{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 机器学习\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 定义\n",
    "随着经验$E$的增长，程序对于任务$T$在度量标准$P$表现良好的准确，我们称之为程序对任务$T$进行了学习。<br>\n",
    "> <font color='red'>这也是机器学习目前的困境所在，只能针对单方面的事物进行学习，还不会变通</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 类别\n",
    "- 回归与分类\n",
    "\n",
    "\n",
    "1. 分类问题：对数据进行类别划分\n",
    "2. 回归问题：对数据进行预测评估\n",
    "> <font color='red'>两者的区别在于结果空间的划分，分类结果严格独立，属于不同的空间，但是回归问题具有同一的空间量度 <br> \n",
    "不过，通过对回归空间的严格区间划分，也可以进行分类问题的求解\n",
    "</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 监督和非监督\n",
    "\n",
    "\n",
    "1. 监督学习：依据既定标准量度进行学习\n",
    "2. 非监督学习：程序自行判断，自行学习并对新数据进行学习成果的展开，不存在人为指定既定标准\n",
    "> <font color='red'>监督学习是对人既定的知识、判断的扩充，非监督学习，完全是程序自我学习的结果。</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 梯度下降\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "沿着梯度反方向进行移动，最终到达最低点。<br>\n",
    "> <font color='red'>针对凸函数</font>\n",
    "\n",
    "对于$f(x)$，梯度下降可以这样进行表示\n",
    "$$\n",
    "\\Large x_{next} = x - \\eta f'(x)\n",
    "$$\n",
    "\n",
    "其中$\\Large \\eta$表示学习率，正是梯度的有效移动率。\n",
    "> <font color='red'>虽然直接求导就是极值，但是对于导数不存在、不可导等一系列问题，梯度下降不存在此类问题，方便普适</font>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 归一化\n",
    "机器对于数据大小敏感，但是对于含义不敏感。<br>\n",
    "$100 > 1$,即时$100nm < 1km$，所以，为了减少误差，一般都需要进行数据归一化处理。\n",
    "\n",
    "## 放缩\n",
    "$$\n",
    "\\Large \\hat x = \\frac{x - x_{min}}{x_{max} - x_{min}}\n",
    "$$\n",
    "这种办法，会按照自身的尺度对数据进行放缩，变换为$\\hat x \\in [0, 1]$的数据。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 偏移\n",
    "$$\n",
    "\\Large \\hat x = \\frac{x - \\mu}{\\sigma}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- $\\Large \\mu$: 样本均值\n",
    "- $\\Large \\sigma$: 样本标准差\n",
    "$$\n",
    "\\Large \n",
    "\\left\\{\n",
    "\\begin{matrix}\n",
    "\\mu =& \\frac{\\sum_i^n x_i}{n} \\\\\n",
    "\\sigma =& \\sqrt{\\frac{\\sum_i^n (x_i - \\mu)^2}{n}}\n",
    "\\end{matrix}\n",
    "\\right.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "虽然没有放缩直观，但是得到的数据样本保持同原样本数据的同分布，对后续计算有极大便利，而且从内在保留数据规律。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 高斯函数\n",
    "\n",
    "$$\n",
    "\\Large f(x) = \\frac{1}{\\sqrt{2\\pi} \\sigma}\\exp\\left(- \\frac{(x - \\mu)^2}{2 \\sigma}\\right)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 线性回归推导\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "{\\Large f(X)} = \\theta_0 + \\theta_1x_1 + \\dots + \\theta_nx_n\n",
    "$$\n",
    "矩阵表示\n",
    "$$\n",
    "\\begin{aligned}\n",
    "{\\Large \\theta} =& \\left[\n",
    "\\begin{matrix}\n",
    "\\theta_n & \\theta_{n-1} & \\dots & \\theta_1 & \\theta_0\n",
    "\\end{matrix}\n",
    "\\right] \\\\\n",
    "{\\Large X} =& \\left[\n",
    "\\begin{matrix}\n",
    "x_n & x_{n-1} & \\dots & x_1 & 1\n",
    "\\end{matrix}\n",
    "\\right] \\\\\n",
    " f(X) =&  \\theta X^T\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于一个已知的函数式而言,$f(x) = \\theta X^T$式没有问题的<br>\n",
    "但是对于一个统计问题，也就是对于$X$全部已知，对$\\theta$一无所知的情况，我们实际求解的应该是$f(\\theta)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 误差引入\n",
    "对于生活中总是存在误差，所以函数这样来表示比较正确\n",
    "$$\n",
    "\\Large Y = f(X) + \\epsilon \\Rightarrow \\epsilon = Y -f(X)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一般来说，误差是服从正太分布的\n",
    "$$\n",
    "\\epsilon  = \\frac{1}{\\sqrt{2\\pi} \\sigma}\\exp\\left(- \\frac{(y_i - f(x_i))^2}{2 \\sigma}\\right)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 极大似然\n",
    "最经常发生的，概率就是最大的。 <br>\n",
    "举个例子，抛硬币，十次里面有六次朝上，那么反面的概率是多少。<br>\n",
    "假设朝上的概率为$p$，朝下的概率为$q$\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "p =& 1 - q \\\\\n",
    "P(6,4) =& p^6q^4 = p^6(1-p)^4 \\\\\n",
    "\\triangledown P(6,4) =& 6p^5(1-p)^4 - 4p^6(1-p)^3 \\\\\n",
    "=& p^5(1-p)^3(6 - 10p) \\\\\n",
    "\\Rightarrow &\\left\\{\n",
    "\\begin{matrix}\n",
    "p_1 =& 0 \\\\\n",
    "p_2 =& 1\\\\\n",
    "p_3 =& \\frac{6}{10} = \\frac{3}{5}\\\\\n",
    "\\end{matrix}\n",
    "\\right.\n",
    "\\end{aligned}\n",
    "$$\n",
    "这样就求解出概率了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "P(y_i|x_i;\\theta) = \\frac{1}{\\sqrt{2\\pi} \\sigma}\\exp\\left(- \\frac{(y_i - f(x_i))^2}{2 \\sigma^2}\\right)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$\n",
    "\\begin{aligned}\n",
    "L(\\theta) =& \\prod P(y|x;\\theta) \\\\\n",
    "=&\\prod_i^m \\frac{1}{\\sqrt{2\\pi} \\sigma}\\exp\\left(- \\frac{(y_i - f(x_i))^2}{2 \\sigma}\\right) \\\\\n",
    "\\end{aligned}\n",
    "$$\n",
    "取对数\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\log(L(\\theta)) =& \\prod_i^m \\log(\\frac{1}{\\sqrt{2\\pi} \\sigma}\\exp\\left(- \\frac{(y_i - f(x_i))^2}{2 \\sigma}\\right) \\\\\n",
    "=& m\\log\\frac{1}{\\sqrt{2\\pi} \\sigma} - \\frac{1}{\\sigma ^2}\\frac{1}{2}\\sum_i^n (y_i - f(x_i))^2\n",
    "\\end{aligned} \n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "所以最后实际上是求取$J(\\theta)$的极小值\n",
    "$$\n",
    "J(\\theta) = \\frac{1}{2}\\sum_i^n(y_i - f(x_i))^2 = \\frac{1}{2}(\\theta X^T - Y)^2 = \\frac{1}{2}(\\theta X - Y)^T(\\theta X - Y)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "导数为\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\triangledown J(\\theta) =& \\triangledown_\\theta\\left(\\frac{1}{2}(\\theta X - Y)^T(\\theta X - Y)\\right) \\\\\n",
    "=& \\triangledown_\\theta\\left(\\frac{1}{2} (\\theta^TX^T - Y)(\\theta X - Y)\\right) \\\\\n",
    "=& \\triangledown_\\theta\\left(\\frac{1}{2}(\\theta^TX^TX\\theta - \\theta^TX^TY - Y^TX\\theta + Y^TY)\\right) \\\\\n",
    "=& \\frac{1}{2}\\left( 2X^TX\\theta - X^TY + (Y^TX)^T\\right) \\\\\n",
    "=& X^TX\\theta - X^TY\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "令导数为0\n",
    "$$\n",
    "\\begin{aligned}\n",
    "X^TX\\theta - X^TY &=& 0 \\\\\n",
    "\\Rightarrow \\theta &=& (X^TX)^{-1}X^TY\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 逻辑回归\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## sigma\n",
    "$$\n",
    "\\Large g(z) = \\frac{1}{e^{-z} + 1}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADuCAYAAACaodTYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU9b3/8ddJJglZCFsIEIYtDAQIhCgD4lJ3ZDUtdjHe21ZKvRSFq/7qVm29drsVu9xbr1iRWy9erYp15VYgiAiKGxBAMIQlCEgSQhYgBMg6M9/fHwmRkIQEyMzJJO/n4zGPnDnf70w+852ZdybfOYtljEFERAIjxO4CREQ6E4WuiEgAKXRFRAJIoSsiEkAKXRGRAHK00K5NG6TdmjJlChkZGXaXIdIUq7kGfdKVoFVSUmJ3CSLnTaErIhJACl0RkQBS6IqIBJBCV0QkgBS6IiIBpNAVv5s9ezbx8fGMHj26yXZjDHfffTcul4uUlBS2bNkS4ApFAkehK343a9asc25Pu3LlSnJycsjJyWHx4sXceeedAaxOJLBa2jni4lx7beN13/se3HUXlJfDtGmN22fNqr2UlMB3vtO4/c474dZbITcXfvCDxu333Qc33wy7d8NPftK4/Re/gBtvhM8/h3vvbdz+u9/BFVfAJ5/AI480bv/znyE1Fd57D37728btzz4LSUnwj3/An/7UuP3FF2HAAHj1VXjmmcbtr78OcXHw/PO1l7OtWAFRUfCXv8Df/964fd262p9//CO8807DtshIWLmydvk3v4E1axq29+oFb7xRu/zww/Dppw3bnU74299ql++9t3YMzzR8OCxeXLs8Zw7s2QPA1UBlZSUPFRR83ff734e8PACG7tnD8927Yz3yCBMff5zS0lIqpk8n8tSphvd/ww3w6KO1y1On1j7HZ77GZsyA+++vXdZrr3H7ebz2PM//LxWhYVSGOKgMcVBlOaj6yzNUOcKpef1Nqj/4kBorlGorFI8VUnt55Od4fT5qMt7Fl7UDj2XhIwSvZeEND8d3+yx8Bnzr1uHbfwAfFj7LwgC+yCjMt76Fz2cwH32EKSjAAAYLY1mYmBjMjZMwGPjoY8yRI/V7bhks6NYNrv4GxgDr18Px4xjLqmsH07MXXHZZ7Q0+/BDqXlv1e3/Fx2MuHVe7vG4tVFUxevb3+NGVQxqP00Xyb+iKtEJ1VRVdIiLqrzudTiorKohsou/ixYtZvHgxT+bkUFNTE7gig5APOOqIojA8huKwKI7uPsbRAzUcPRTO8SGTOB4aQZmjCydDwzkRGsGpZ7dyymOoqIyjeuL9je9wcWbdQl8Y+b3G7a9tq1voB0P6NW5/d0/dQgI4EwgxPkKMIaQu+kI//QrLAos+WL3jOB2ZIRgsKwRr+yEsy8IK7QM9e2Kdsb+s5QjF2llUuxzeB3r0ql2u62OFhmHtrduZJqIPOLxf3xYDRMC+I7UrIvtChI/wwhPND+5FsFo4iLl2A5Y2ceDAAWbMmEFWVlajtunTp/Pwww9z1VVXAXDDDTfw+9//nnHjxp3zPt1uN5mZmefs09GdrPKwr/gk+0tOsa/4FLlHy8krrSD/WAWFZZV4fI3fwqEhFt0iw+geGUbXyDBiuziIiXAQFe4gJiKUqAgHkWGhRIWHEhEWShdHCBFhoYSHhhDhCCG87hIWGkJYqEVYaAiOkNqfoSEWoSEWjrqfISEWoVbdcv1PsKxm95LtKJp9gPqkK7ZzOp3k5ubWX8/LyyMhIcHGitqn4xU1fJ5bytaDx8g+VMbOw2XkHq2ob7csSOgWSf/ukVw2pCd9u3WhT2wX+sRGEBcTQa+YCHpGhxPbxdEZQq/dUuiK7dLS0li4cCHp6els2LCBbt260a9fE/+edjInqzxs2HeEj/aW8MneI+yu+3fXsmBIXDQp/btzq3sArviuDO0dzYCeUXQJC7W5ammJQlf87rbbbmPdunWUlJTgdDr51a9+VT8fO3fuXKZNm8aKFStwuVxERUWxZMkSmyu2z7FT1azOLmRlVgEf7S2hxmuIcIQwYUhPZqT045KBPRg7oBtdu4TZXapcIM3pStDqKHO6Xp9hfU4xSzfm8t7OQjw+Q//ukUwd3ZfrRsQzblAPfYINPprTFWlvyqs9vLzhIEs+PkB+aQU9o8P50ZWDSRvbn9H9YzXv2kEpdEUC7ERlDUs+PsCSj/dzrLyGCUN68si0kUwa1Ydwh/ZX6ugUuiIB4vUZXt+cyx9W7aHkZBU3jIjnruuGMm5QT7tLkwBS6IoEwOe5pTzy5hdkF5QxblAP/nq7m9QB3e0uS2yg0BXxo2qPj4Xv5/D0ui+J7xrBf912CTen9NN8bSem0BXxky+LT3LP0q1k5Zfx7UudPJY2ilht6tXpKXRF/GDtriLufmUrYY4Qnv3BOCYn97W7JGknFLoibcgYw6IP9vH7VbsY1S+WxT900797U4fukc5KoSvSRrw+w8/f+oKlm3KZntKPP35nLJHh2qlBGlLoirQBj9fHA69v562t+cy7bij335SkL8ukSQpdkYtU4/Vx76ufs3x7AfffNJz51w+zuyRpxxS6IhfB5zP8v7rA/fm0kfzL1Yl2lyTtnPY5FLkIT2Ts4p3tBfxs6ggFrrSKQlfkAr342Vc8++E+fjBxED9R4EorKXRFLsD7uwp5bFkWN4yI57GbR+lLM2k1ha7Ieco9Ws49Sz9nVEIsT/3TJThC9TaS1tOrReQ8VHt8zH95CwDP/PM4osL1XbScH71iRM7D7zN2sS3vOIu+fykDekbZXY4EIX3SFWml97IL+etH+7n98kFMGa0TZ8qFUeiKtMKxU9U89MZ2RvWL5eFpI+0uR4KYphdEWuG3y3dyvKKGv91xmU4SKRdFn3RFWrA+p5g3tuQx95qhjOwXa3c5EuQUuiLnUF7t4ZG3viAxLpr517vsLkc6AE0viJzDf67eQ+7RCl6dM1HTCtIm9ElXpBlfFp9kyccHuG3CAC5L7GV3OdJBKHRFmrFg5S66hIVy301JdpciHYhCV6QJn+07wursQu68dihxMRF2lyMdiEJX5Cw+n+G3y7NJ6NaFH181xO5ypINR6IqcZdm2fLLyy3hgSpK+PJM2p9AVOUOVx8sfV+1hTP9ufHNsf7vLkQ5IoStyhjc255NfWsH9k5MICdExcqXtKXTF7zIyMkhKSsLlcrFgwYJG7cePH+fmm29m7NixJCcns2TJEhuqrD3B5F/W7WXsgO5cPSzOlhqk41Poil95vV7mzZvHypUryc7O5pVXXiE7O7tBn6effppRo0axbds21q1bx3333Ud1dXXAa31raz55xyq4+3qXzgQhfqPQFb/auHEjLpeLxMREwsPDSU9PZ9myZQ36WJbFiRMnMMZw8uRJevbsicMR2J0lPV4ff1m7l+SEWK4fER/Q3y2di0JX/Co/P58BAwbUX3c6neTn5zfoM3/+fHbu3ElCQgJjxozhySefJCSk6Zfm4sWLcbvduN1uiouL26zOd7YXcOBIOf+qT7niZwpd8StjTKN1Z4faqlWrSE1N5dChQ3z++efMnz+fsrKyJu9vzpw5ZGZmkpmZSe/evdukRp/PsHDtXpL6dOWmUX3b5D5FmqPQFb9yOp3k5ubWX8/LyyMhIaFBnyVLlnDLLbdgWRYul4shQ4awa9eugNW4bk8Re4tOcue1Q7XFgvidQlf8avz48eTk5LB//36qq6tZunQpaWlpDfoMHDiQNWvWAFBYWMju3btJTEwMWI1LPj5An9gIpqfoFDzifzq0o/iVw+Fg4cKFTJ48Ga/Xy+zZs0lOTmbRokUAzJ07l0cffZRZs2YxZswYjDE88cQTxMUFZpOtPYUnWJ9TwgOTkwjTqdQlAKym5tzOcM5GETu53W4yMzMv6j4efvML3tySx6cP30DP6PA2qkyEZuep9KddOq1jp6p5c0seMy/pr8CVgFHoSqf1yqaDVHl8zLpysN2lSCei0JVOqcbr44VPvuJKVy9G9NXJJiVwFLrSKa3dVcThskpuv3yw3aVIJ6PQlU5p6aZc4rtGaJdfCTiFrnQ6BccrWLe7iO+6nTi0mZgEmF5x0um8lpmHz8Ct7oF2lyKdkEJXOhWfz/DqplyucsUxsFeU3eVIJ6TQlU5l/d4S8ksrSJ8woOXOIn6g0JVOZenGg/SICmPSqD52lyKdlEJXOo2Sk1Wszi7k25c6iXDoLL9iD4WudBr/2HYIj8/wXbemFsQ+Cl3pNN7ems+ofrEk9e1qdynSiSl0pVP4svgk2/KOc8ul/e0uRTo5ha50Cm9vzSfEgrSxCS13FvEjha50eD6f4a2t+VzpiiM+tovd5Ugnp9CVDm/zwWPkHatg5iWaWhD7KXSlw3tzSz6RYaFMTtaZfsV+Cl3p0Ko8XpZvP8Tk5D5ER+iUgGI/ha50aB/sLqas0sO3NLUg7YRCVzq0d7YX0CMqjCtdgTm7sEhLFLrSYVXWeHlvZyFTRvfV6dWl3dArUTqsdbuLKK/2MiNF2+ZK+6HQlQ7rH9sL6BUdzmVDetpdikg9ha50SOXVHt7fWcSU0X11Sh5pV/RqlA5p7a5iKmo0tSDtj0JXOqR3th8iLiaCCZpakHZGoSsdzqkqD+/vKmLamL6Ehlh2lyPSgEJXOpw1u4qo8viYPqaf3aWINKLQlQ5nVdZh4mIicA/W1IK0Pwpd8buMjAySkpJwuVwsWLCgyT7r1q0jNTWV5ORkrrnmmgv+XZU1XtbuLuKm5D6aWpB2SUcAEb/yer3MmzeP1atX43Q6GT9+PGlpaYwaNaq+T2lpKXfddRcZGRkMHDiQoqKiC/5963NKKK/2MkVHFJN2Sp90xa82btyIy+UiMTGR8PBw0tPTWbZsWYM+L7/8MrfccgsDBw4EID4+/oJ/X0bWYWK7OLh8aK+LqlvEXxS64lf5+fkMGPD12XedTif5+fkN+uzZs4djx45x7bXXMm7cOF544YVm72/x4sW43W7cbjfFxcUN2mq8Pt7bWciNo/roWAvSbml6QfzKGNNonWU1nGv1eDxs3ryZNWvWUFFRweWXX87EiRMZPnx4o9vOmTOHOXPmAOB2uxu0fbbvCMcrajS1IO2aQlf8yul0kpubW389Ly+PhISERn3i4uKIjo4mOjqaq6++mm3btjUZuueSkXWYyLBQrh7eu01qF/EH/Q8mfjV+/HhycnLYv38/1dXVLF26lLS0tAZ9vvnNb7J+/Xo8Hg/l5eVs2LCBkSNHntfv8foMq3YUct2I3nQJC23LhyDSpvRJV/zK4XCwcOFCJk+ejNfrZfbs2SQnJ7No0SIA5s6dy8iRI5kyZQopKSmEhIRwxx13MHr06PP6PVsPHqPkZBVTRmuHCGnfrKbm3M5wzkYRO7ndbjIzMwH49+XZPP/JAbY8OomuXcJsrkyEZjcS1/SCBD1jDO9mF3LF0DgFrrR7Cl0JejlFJ/nqSDk3JfexuxSRFil0Jei9u+MwADeOVOhK+6fQlaC3OruQ1AHd6RPbxe5SRFqk0JWgVnC8gm15xzW1IEFDoStB7b3sQgBuGqXQleCg0JWg9m52IYlx0QztHWN3KSKtotCVoOX1GT798giTkvs0Op6DSHul0JWgdaLSg8dnNLUgQUWhK0GrrLKGuJhwUgf0sLsUkVZT6EpQqvb4OFHp4YYROi2PBBeFrgSlz/YdwWcMkzS1IEFGoStBaXV2ISGWxVXD4uwuReS8KHQl6BhjeG9nITERDh07V4KOQleCTlZ+GQXHK4mN1OGgJfgodCXorN5ZSIiFDuMoQUmhK0FndXYh7kE9cWirBQlCCl0JKrlHy9lZUMaNo+LtLkXkgih0Jai8t7P2ADeTRuk06xKcFLoSVFZnF+KKj2FIXLTdpYhcEIWuBI3S8mo27D/KZB07V4KYQleCxvu7ivD6jKYWJKgpdCVovLujkD6xEaT072Z3KSIXTKErQaGyxsuHOcVMGtWHEG0qJkFMoStB4eO9JZRXe7lJUwsS5BS6EhTe3VFI1wgHExN72V2KyEVR6Eq75/XVHuDm2hHxhDv0kpXgplewtHtbDx7jyKlqnZZHOgSFrrR772YXEhZqcW1Sb7tLEbloCl1p14wxZGQd5oqhcTqqmHQICl1p13YWnODg0XKmjtZWC9IxKHTF7zIyMkhKSsLlcrFgwYJm+23atInQ0FBef/31r2+74zAhFtyo+VzpIBS64lder5d58+axcuVKsrOzeeWVV8jOzm6y30MPPcTkyZMbrF+VdZjxg3sSFxMRqJJF/EqhK361ceNGXC4XiYmJhIeHk56ezrJlyxr1e+qpp/j2t79NfPzXx8ndV3yS3YUnmKKpBelAFLriV/n5+QwYMKD+utPpJD8/v1Gft956i7lz5zZYv2pH7bFzJyd/HbqLFy/G7XbjdrspLi72Y+Ui/qHQFb8yxjRaZ1kNj51w77338sQTTxAa2vDMvhlZBYx1diOhe2T9ujlz5pCZmUlmZia9e2sTMgk+Op2q+JXT6SQ3N7f+el5eHgkJCQ36ZGZmkp6eDkBJSQkrVqzghNfBtrwwHpySFNB6RfxNoSt+NX78eHJycti/fz/9+/dn6dKlvPzyyw367N+/v3551qxZzJgxgxP9xsLWbKYkaz5XOhaFrviVw+Fg4cKFTJ48Ga/Xy+zZs0lOTmbRokUAjeZxT1uZdZjhfWJI7B0TyHJF/M5qas7tDOdsFPGHwrJKJj6+hntvGM49Nw5rtp/b7SYzMzOAlYm0WrMHfdYXadLurPiiAGNgeko/u0sRaXMKXWl3lm8vYETfrrjiNbUgHY9CV9qVQ6UVZH51jBn6lCsdlEJX2pUVXxQAMD0loYWeIsFJoSvtyjvbC0hOiGVIXLTdpYj4hUJX2o3co+V8nlvKDH3KlQ5MoSvtxsqsuqmFMZrPlY5LoSvtxj+2FZDi7MbAXlF2lyLiNwpdaRf2Fp3gi/zjfCu1v92liPiVQlfahbe25hMaYnHzWM3nSsem0BXb+XyGt7ce4ipXHL276gwR0rEpdMV2mw4cJb+0glsu1dSCdHwKXbHdW1vziQoPZZJOPimdgEJXbFVZ42X5FwVMSe5LVLiONCodn0JXbPX+riJOVHqYqakF6SQUumKrN7fk07trBFcMjbO7FJGAUOiKbYpOVLJudxEzL+lPaEizx3wW6VAUumKbNzbn4/EZbh0/oOXOIh2EQldsYYzh1U0HmTCkJ0N1HjTpRBS6YotP9x3hwJFy0vUpVzoZha7Y4tVNucR2cTBNRxSTTkahKwFXWl7NyqzDzLykP13CQu0uRySgFLoScG9uyafa4yN9wkC7SxEJOIWuBJQxhqWbDjJ2QHdG9ou1uxyRgFPoSkB9+uUR9hSe5J8v06dc6ZwUuhJQ//PxfnpFh5Om4+ZKJ6XQlYA5UHKKNbuK+OeJg/QFmnRaCl0JmOc/OYAjxOL7EzW1IJ2XQlcC4kRlDa9vzuPmlATiu3axuxwR2yh0JSD+npnHySoPP7pyiN2liNhKoSt+t3zFSn6z9EMo+ZLlf3umUftLL71ESkoKKSkpXHHFFWzbts2GKkUCQ6ErfuX1epn/h+chJo6n//UWXnnlFbKzsxv0GTJkCB988AHbt2/n0UcfZc6cOfYUKxIACl3xq882bCBs7AxG9O3K1DH9SU9PZ9myZQ36XHHFFfTo0QOAiRMnkpeXZ0epIgGh0BW/WplVSHWXnsy/3kVIiIXT6SQ/P7/Z/s899xxTp05ttn3x4sW43W7cbjfFxcX+KFnEr3QmQPEbn8/wXoGDaE8ZU0d/fTQxy2r6LBFr167lueee46OPPmr2PufMmVM//eB2u9u2YJEA0Cdd8ZvVOws5XBFCbN6n9afjycvLIyGh8d5o27dv54477mDZsmX06tUr0KWKBIxCV/zCGMNT7+cwqFcUhzetYP/+/VRXV7N06VLS0tIa9D148CC33HILL774IsOHD7epYpHA0PSC+MXyLwrIyi/jj98dS1TyfzF58mS8Xi+zZ88mOTmZRYsWATB37lx+/etfc+TIEe666y4AHA4HmZmZdpYv4jeWMeZc7edsFGlKlcfLjf/xAdHhDpbf/Q2/nenX7XYrnKW9avZFr+kFaXP/+8kBco9W8Ivpo3RqdZGzKHSlTR07Vc1T7+/l2qTeXDUszu5yRNodha60qSfX5HCqysMj00baXYpIu6TQlTaTU3iCv332FbeOH8jwPl3tLkekXVLoSpvw+QwPvbGdmC4O7rtJm32JNEehK23ibxu+YsvBUv5txijiYiLsLkek3VLoykU7VFrBEyt38Y1hccy8pL/d5Yi0awpduSjGGB59Owufgd/NHNPscRVEpJZCVy7Ka5l5rNlVxH03DWdAzyi7yxFp9xS6csH2FJ7g3/4viyuG9tJpeERaSaErF6S82sO8l7YQE+Hgz+mp2vNMpJV0wBu5II8t28He4pO8OPsynd1X5Dzok66ct5c2fMVrm/OYf51Lu/qKnCeFrpyXtbuKePTtLK5L6s09NwyzuxyRoKPQlVbLyj/OvJe3MCohloX/dCmOUL18RM6X3jXSKrlHy/nR85voERXO/9w+nugIfR0gciEUutKi/SWnuPXZT6n2+Fjyo/HEx+qLM5ELpY8rck57i07wT/+9AY/P8Mq/TNTRw0QukkJXmpWVf5xZSzYCFkvnKHBF2oKmF6RJy7cX8J1FnxAWGsKrP1HgirQVfdKVBnw+w3++t4en3t/LuEE9eOb7l2rnB5E2pNCVegXHK3jw9e2szynhe24nv/nWaCIcoXaXJdKhKHQFYwxvf57Pvy3bgcdr+N3MMdw2YYAO0yjiBwrdTm5v0Un+fXk2a3cXM25QD/703bEMjou2uyyRDkuh20mVllfz5JocXvz0KyLDQvnF9JH86MohOlqYiJ8pdDuZorJK/vrRfl767CsqarykTxjITycN13nNRAJEodsJGGPYmlvK0o0HeXvrITw+HzNSEph3nYukvtoUTCSQFLodWN6xcjKyDvNaZh67C08QFR7Kd9xOfnJ1IoN6ad5WxA4K3Q7E6zN8kX+cj3KKeTe7kO15xwFIcXbjdzPHcPPYfnTtEmZzlSKdm0I3iFXWePki/zhbvjrGloPH+GzfUY5X1AAw1tmNn00dwdTRffWpVqQdUegGgcoaLweOnGJ/8Sn2lZxiZ0EZuw6fYF/xSXymts/AnlFMTu7Dla44rnTF6YsxkXZKoWujyhovpeU1HDlVxdFT1ZScrKKorIqiE1UcLqsk/1gF+aUVFJ+oanA7Z49IRvSNZerovqQ4u3PJwO7tOmQzMjK455578Hq93HHHHfzsZz9r0G6M4Z577mHFihVERUXx/PPPc+mll9pUrYh/BX3oGmPqfoIBfMbULdf+PN3mM6buUnsb35nrfOA1Bp+v9rrHV7vsNQaP1+D11a7zeH3UeA01Ph81ntrlaq+Xao+Pao+PqrpLRbWXyhovFTVeyqu9nKryUF7t5USVh5OVNZyo9FBaUUO1x9fkY4oMC6Vvty4kdO/CdUm96d89iiG9o0mMi2ZwXDQxQXQAca/Xy7x581i9ejVOp5Px48eTlpbGqFGj6vusXLmSnJwccnJy2LBhA3feeScbNmywsWoR//HLu3f34RN88+mP6q+fDr+zmSaunBmWZzbVh+s57q+9CLFqgzMyvPYSHe4gMjyUbpFhOLtH0rWLg26RYcRGhtE9Koxe0eH0jI6gZ3Q4fWIjiIlwdJhdcDdu3IjL5SIxMRGA9PR0li1b1iB0ly1bxg9/+EMsy2LixImUlpZSUFBAv3797CpbxG8sc44EmzJliikpKTnvO/V4DSUnqyivKCcqMuo8qrmgpq/7NBFU1hkLp5dPnTxFTEw0WFb9OuuMzhYWlvX1equu0bJOXz+9XPfTghDLIsRqeNvzVVxcTO/evS/sxn50MXUdO3aMsrIyBg0aBMCRI0c4deoUAwcOrO+zd+9e+vbtS0xMDAB79uzB6XQSFdX4tVNcXMzp12RVVRWpqakXVJc/dcTn0Z86Yl2bN29eZYyZ0mSjMeZcl4sybty4i70Lv1Bd5+di6vr73/9ufvzjH9dff+GFF8z8+fMb9Jk2bZpZv359/fXrr7/eZGZmtnjfUVFRF1yXP3XE59GfOmhdzeaqDmIufuV0OsnNza2/npeXR0JCwnn3EekoFLriV+PHjycnJ4f9+/dTXV3N0qVLSUtLa9AnLS2NF154AWMMn332Gd26ddN8rnRYfv0afM6cOf68+wumus7PxdTlcDhYuHAhkydPxuv1Mnv2bJKTk1m0aBEAc+fOZdq0aaxYsQKXy0VUVBRLlixp1X3HxcVdcF3+1BGfR3/qbHWd84s0ztrAQKQ9cbvdZGZm2l2GSFOa/Tpd0wsiIgGk0BURCaCLCt3XXnuN5ORkQkJCGv2b9/jjj+NyuUhKSmLVqlVN3v7o0aNMmjSJYcOGMWnSJI4dO3Yx5TTr1ltvJTU1ldTUVAYPHtzstp2DBw9mzJgxpKam4na7/VLLmX75y1/Sv3//+tpWrFjRZL+MjAySkpJwuVwsWLDA73U98MADjBgxgpSUFGbOnElpaWmT/QIxXi09dmMMd999Ny6Xi5SUFLZs2eKXOs6Wm5vLddddx8iRI0lOTubJJ59s1GfdunV069at/vn99a9/HZDaWnpe7Biz3bt3149DamoqsbGx/PnPf27QJ1DjNXv2bOLj4xk9enT9utZmUZu8F8+1PVlLG6JlZ2ebXbt2mWuuucZs2rSpfv2OHTtMSkqKqaysNPv27TOJiYnG4/E0uv0DDzxgHn/8cWOMMY8//rh58MEHL3yruFb66U9/an71q1812TZo0CBTXFzs9xpOe+yxx8wf/vCHc/bxeDwmMTHRfPnll6aqqsqkpKSYHTt2+LWuVatWmZqaGmOMMQ8++GCzz4u/x6ulxz5u3DizfPlyM2XKFOPz+cynn35qJkyY4Ld6znTo0CGzefNmY4wxZWVlZtiwYY2el7Vr15rp06cHpJ4ztfS82DVmp3k8HtOnTx9z4MCBBusDNV4ffPCB2bx5s0lOTq5f15osOs/3YrO52lLotuoCrAPcZ1x/GHj4jOurgMubuN1uoF/dcj9gd1vUc446LSxGgW4AAAOMSURBVCAXGNZM+wEgzp81nPX7fgnc30Kfy4FVzY1tAGqcCbxkx3i19NiBDOBZ4LamXlOBvADLgElnrbsWeMeGWs75vNg9ZsBNwMdNrA/YeAGDgaymxqC5LGqr96K/5nT7Uxtup+XVrTtbH2NMAUDdz3g/1XPaN4BCY0xOM+0GeNeyrM2WZQVqO5b5lmVttyzrfyzL6tFEe2vH0l9mAyubafP3eJ3zsZva3SztHh8syxoMXAI0dZSeyy3L2mZZ1krLspIDVFJLz4vdY5YOvNJMmx3jBa3LojYZtxa307Us6z2gbxNNPzfGLGvuZk2s8+vmZ62s8zaaf7IBrjTGHLIsKx5YbVnWLmPMh/6qC3gG+A21Y/Mb4E/UhlyDu2jithc9lq0ZL8uyfg54gJeauZs2H6+zy2xi3dmPPeCvtQa/3LJigDeAe40xZWc1bwEGGWNOWpY1DXgbGBaAslp6XmwbM8uywoE0aj8lns2u8WqtNhm3FkPXGHPj+d4ptX8BBpxx3QkcaqJfoWVZ/YwxBZZl9QOKLuB3AS3XaVmWA7gFGHeO+zhU97PIsqy3gAnARYVIa8fPsqz/Bt5poqm1Y9mmdVmWdTswA7jB1P0v1cR9tPl4naU1j90v49MalmWFURu4Lxlj3jy7/cwQNsassCzrL5ZlxRljzv8oUuehFc+LbWMGTAW2GGMKz26wa7zqtCaL2mTc/DW98H9AumVZEZZlDaH2r9XGZvrdXrd8O7XzYv5yI7DLGJPXVKNlWdGWZXU9vUztvFOWH+uh7sk9bWYzv28TMMyyrCF1nxLSqR03f9Y1BXgISDPGlDfTJxDj1ZrH/n/AD61aE4Hjp/9N9Cer9pB2zwE7jTH/0UyfvnX9sCxrArXvtyN+rqs1z4stY1an2f827RivM7Qmi9rmvXiRk9EzqU3/KqCQhpPMPwe+pHaCeuoZ6/9K3ZduQC9gDZBT97OnHyfOnwfmnrUuAVhRt5wIbKu77KD232x/T+a/CHwBbK978vqdXVfd9WnAnrrxDERde6mdu/q87rLIrvFq6rEDc08/l9T+y/d0XfsXnPGFrp/H6Cpq/7XcfsY4TTurtvl1Y7MN+Ay4IgB1Nfm8tJMxi6I2RLudsS7g40Vt6BcANXX59ePmssgf78WWdgMWEZE2pD3SREQCSKErIhJACl0RkQBS6IqIBJBCV0QkgBS6IiIBpNAVEQmg/w/Ty/Mr0kMw3gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "\n",
    "def sigma(x):\n",
    "    return 1 / (np.power(np.e, -x) + 1)\n",
    "\n",
    "def center(ax):\n",
    "    ax.spines['right'].set_color('none')\n",
    "    ax.spines['top'].set_color('none')\n",
    "    ax.xaxis.set_ticks_position('bottom')\n",
    "    ax.spines['bottom'].set_position(('data',0))\n",
    "    ax.yaxis.set_ticks_position('left')\n",
    "    ax.spines['left'].set_position(('data',0))\n",
    "\n",
    "X = np.linspace(-10, 10, 100)\n",
    "Y = sigma(X)\n",
    "top = np.ones(X.shape)\n",
    "axes = plt.subplot(111)\n",
    "center(axes)\n",
    "plt.plot(X, top, color='red', linestyle=\"--\")\n",
    "plt.plot(X, Y)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先经过线性变换得到基础表达式$f(x)$，然后通过$g(z)$函数进行空间映射\n",
    "$$\n",
    "\\begin{aligned}\n",
    "f(x) =& \\theta X^T \\\\\n",
    "g(\\theta, x) =& g(f(x)) =\\frac{1}{e^{-f(x)} + 1}\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "二分类而言\n",
    "$$\n",
    "\\begin{aligned}\n",
    "\\left\\{\n",
    "\\begin{matrix}\n",
    "true & g(x) \\ge 0.5 \\\\\n",
    "false & g(x) < 0.5\n",
    "\\end{matrix}\n",
    "\\right.\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了统一两次的定义，我们这样来表示概率\n",
    "$$\n",
    "\\Large P(y|x;\\theta) = g(\\theta, x)^{y} (1-g(\\theta ,x))^{1-y}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 极大似然\n",
    "$$\n",
    "L(\\theta) = \\prod_i^n g(\\theta, x_i)^{y_i} (1-g(\\theta ,x_i))^{1-y_i}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 取对数\n",
    "$$\n",
    "\\log(L(\\theta)) = \\sum_i^n\\left(y_i\\log g(\\theta,x_i) + (1 - y_i)\\log (1 - g(\\theta, x_i)))\\right)\n",
    "$$\n",
    "最后求解得到和线性回归最小而乘法相同。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 损失函数\n",
    "我们所说的梯度下降，就是使用在损失函数上面的，损失最小的，也就最大可能了。<br>\n",
    "也就是我们通过最大似然函数得到的等值中的负项。<br>\n",
    "理论上通过梯度下降可以使下降最快，最好，但是工业上面可以遍历，手工试一下。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd5xU1f3/8deHpcjaAQuiLBa6HexoVLDEhopGzaJoiMQCitEoSNSvGGzxZzAqRlSUwNoNarAiBrvCIqAILFgAEQUErKAIfH5/3BmZXebOTt2dnX0/H495zM6de++c+0jy4eRzzvkcc3dERKSwNKjtBoiISPYpuIuIFCAFdxGRAqTgLiJSgBTcRUQKkIK7iEgBqja4m9koM1tqZjNjjv3dzOaY2QdmNs7Mtor5brCZfWxmFWZ2TK4aLiIi4ZLpuT8EHFvl2ARgd3ffE5gLDAYws07AmUDnyDUjzKwoa60VEZGkVBvc3f11YEWVYy+7+9rIx3eBHSN/9wQedfef3f0z4GNg/yy2V0REktAwC/f4A/BY5O9WBME+alHk2EbMrB/QD2DTTTft0qFDhyw0RUSk/pg6derX7r5NvO8yCu5mNgRYC5RFD8U5LW59A3cfCYwE6Nq1q5eXl2fSFBGResfMFoR9l3ZwN7M+wAlAd99QoGYRsFPMaTsCi9P9DRERSU9aUyHN7FjgKuAkd18V89WzwJlm1sTMdgbaApMzb6aIiKSi2p67mT0CHA60MLNFwHUEs2OaABPMDOBdd7/A3T8ys8eBWQTpmovdfV2uGi8iIvFZPpT8Vc5dRCR1ZjbV3bvG+04rVEVECpCCu4hIAVJwFxGpixaEzoIEFNxFROqWn36CG26AahZ+KriLiNQV48dD585w7bVw4okJT1VwFxHJdx9/DCecEAT0Jk3glVfg8ccTXqLgLiKSr1atgr/+Neitv/Ya3HYbzJgB3btXe2k2CoeJiEg2ucO4cXDZZbBwIfTuDbfeCi1bJn0L9dxFRPJJRQUccwz06gVbbQWvvw5jxqQU2EHBXUQkP3z/PVx1FeyxB0yeDP/8J0ydCocemtbtFNxFRKpRVgZt2kCDBsF7WVl1V6TAHR59NJjaeOutQQpm7lwYMAAapp85V85dRCSBsjLo1y8Y24Rg7VC/fsHfpaUZ3nzmzCCIT5oE++4LTz4JBx2U4U0D6rmLiCQwZMiGwB61alVwPG3ffhsMlu69N3zwAdxzT5CKyVJgBwV3EREgPPWycGH888OOJ7R+Pfz739C+PdxxB/zxj0EK5oILoKgozZbHp7SMiNR7iVIvrVvHL+PSunWKPzJtGvTvD2+/DQccEKw27Rq3Wm9WqOcuIvVeotTLsGFQXFz5u+Li4HhSVqyAiy4KAvm8eTBqVBDgcxjYQcFdRCRh6qW0FEaOhJISMAveR45MYjB1/Xq47z5o1w7uvRcuvjiYw37eeUHuJ8eUlhGReq+61EtpaYozYyZPDlIwU6ZAt25w112w115ZaWuy1HMXkXov49RL1LJlcP75cOCB8PnnwcrS11+v8cAOCu4iIhulXpo3h6ZN4eyzk1y0tG4d3H13kIJ56CH485+DFEzv3sENa4GCu4gIQYCfPz/obK9eDcuXB4tHozNnQgP8W28Fg6P9+8M++wRVG2+7DbbYoiabvxEFdxGRGEkvWvrqK+jTJ8ipf/11UF994kTo1KnG2pqIgruISIxqFy398gsMHx4sRHrkERg8GGbPhtNPr7UUTDwK7iIiMcIWJ7VuzYYaMJddBgcfHNSGufFG2GyzmmxiUhTcRaTeildyIN7MmV03+YJJLc+CI46AH36Ap5+G558PBlDzlIK7iNRL0ZIDCxZUHjiFDTNnGrOGm7a6hdnenjbTxsF118GsWdCzZ16lYOLRIiYRqZcSDZzOnw+l204IyvFWVAQbUw8fDrvsUittTUe1PXczG2VmS81sZsyxZmY2wczmRd63jvlusJl9bGYVZnZMrhouIpKJ0KqOCxYEW9wdfXQwf/255+DZZ+tUYIfk0jIPAcdWOTYImOjubYGJkc+YWSfgTKBz5JoRZpbdOpYiIllQdeC0CT8xhL8x2zrCCy8EyfeZM+G442qngRmqNri7++vAiiqHewKjI3+PBk6OOf6ou//s7p8BHwP7Z6mtIiJZEztwehzPMZPd+RvXsHS/42HOHLj6amjSpHYbmYF0B1S3c/cvASLv20aOtwI+jzlvUeSYiEheKS2Fh2/4hAlNT+Q5ToCGjZg4aAIl7z2RRrH2/JPt2TLxho897olm/cys3MzKly1bluVmiIgksGoVXHstPa/uTI+iSfD3v7PbjzPoflOPlG6T042zM5RucF9iZi0BIu9LI8cXATvFnLcjsDjeDdx9pLt3dfeu22yzTZrNEBFJgTuMGxeUCLjhhmDgtKICrrgCGjdO6VZhUynzJcCnG9yfBfpE/u4DPBNz/Ewza2JmOwNtgcmZNVFEJAsqKuDYY+HUU4OiXq+9FkTiHXZI63Y52Tg7i5KZCvkI8A7Q3swWmVlf4GbgKDObBxwV+Yy7fwQ8DswCXgQudvd1uWq8iEi1fvgBBg2CPfaAd98N5qu//z4cdlhGt83qxtk5kMxsmbPcvaW7N3L3Hd39AXdf7u7d3b1t5H1FzPnD3H1Xd2/v7i/ktvkiUt+F5r3d4bHHoEMHuOWWYAR17lzKWlxKm90aZpwnT1iDJg+o/ICI1Flhee/xt3wE3bvDmWfCdtsFG1I/+CBlr2yXtTx51nZvyhEFdxGpc6K99d69K+e9N+c7blj1Z44dtBdMnw733BPsZ3rQQUB28uTR3z777GC3pubNU9w4u4aotoyI1CnR3nrlIO30Zix/5y9sy1Lu53z6zR0GLVpUujbTPHnV316+POitjxmTP0E9Sj13EakTwnrrezGdNziUMZzDfNqwP5O5seTejQI7ZJ4nz/cZMrEU3EUk78Xm1qO2YiV30p+pdKE9FfTlfg7mbWYXdw3Ne2eaJ8/3GTKxFNxFJO/F9piN9fTlfubSjgu5hxFcRDvmMoq+tC5pkDDvXVq6oVZ7OnnyfJ8hE0vBXUTyXrRn3IVy3uEg7ud8KmhPF6ZyCXeypnhrxo6N1GGvJlCXlgbnrV+f3Pmx8n2GTCwFdxHJe3u1+pp76cdk9qc1C+nNGA7lDWawd43OUsm051+TFNxFpMYlW3Dr4THr+Gvze5i4qB1/YBTDGUh7KiijN8XFlnRvPZsy6fnXJE2FFJEaVXU6YezepbGB8qXr3qbTDf35vU/jVY5gAHcy2zrjHvSYhw3L38CaD9RzF5EaETaVEapMJ1yyBM49l2OGHkILX8oZPEp3JjKLDYE9n3vM+UI9dxHJufgLjyr7YsFauONuuPZaWL2amxnE3xjCj2xW6bx8nHaYj9RzF5GEMtmQIlFvPdZhvMaHjfaBgQPhwAPhww/5V8lNGwV2SDztMKyt+bypRs64e62/unTp4iKSf8aOdS8udg/KbAWv4uLgeDrXVn3twCIv4yx38O9blLiPG+e+fn1avx3vfLPK76k+Q74Dyj0krtZ6YHcFd5G8VVISPyiXlKR/Lbg34me/glv9Ozbzn2jiH5x8jfuPP250j7Fjg/uYBe+JAnKi30v3GfJdouCutIyIhMpkuX3YOT2YwAz24u9cyTf7HkmTT2bxwWlDadOpeKO0SSrTDlPNxRd67l7BXURCZbLcvuo5O7GQJziNCRxN04a/8L/Lx7PT1Gcoe2eXrNRYT7UEQD6WDMgmBXcRCZXJcvvotU34iasZxhw6cBzPM/30v9Hm+5kccdvxQPYqLcZra5h8LRmQVWH5mpp8Kecukr9SyXtX9erl4/2zhru6gz9ffKqPGz5/o3OqDnbGDoam29Z4g6jRz6k+Qz4jQc7dgu9rV9euXb28vLy2myEi2fLpp8G0xv/+F9q3hzvvhKOOintqmzaVS/lGRRcrpausLOj9L1wYpGAKcUWrmU11967xvlNaRkSyZ/VquO466NQJXn012Jj6gw9CAzvkrtJiXakBkysK7iKSOXd4+ukgqA8dCqeeChUVcOWV0LhxwkurVlps3jzYm/Tss+vRgqMcUHAXkczMnQvHHQennAKbbQb/+x88/DC0apX0LaK97DFjgs7/8uWZzZwRBXcRSdePP8LgwbD77qx57W2u33o4jWe+T5tzD087GNelPUrznQqHiUhq3OGJJ+Dyy2HRIj7tdg49pt7CZyu3B8JL+CajLu1Rmu/UcxeR5M2aBT16wBlnsKJoG07d7i12fXM0n63evtJp6fa269IepflOwV1Eqvfdd0FPfa+9YNo0Jve5m5KlUxi35ODQS9LpbdelPUrznYK7iIRzh7Fjg7nq//gHnHcezJ3L7yZdxA+rixJemk5vuy7tUZrvMsq5m9llwB8BBz4EzgOKgceANsB84HfuvjKjVopIzZsxA/r3hzffhP32g2efDd6pvleeSW+7tFTBPBvS7rmbWSvgEqCru+8OFAFnAoOAie7eFpgY+SwidcXKlTBgAOy7L8yZA/ffD++++2tgh8S9cvW280OmaZmGQFMza0jQY18M9ARGR74fDZyc4W+I1Hu52kko9r47l6znnX6jghTMiBFwwQXBQqS+fYMTYoTlxseODb4bMqSe7XqUj8KKziTzAi4FfgCWAWWRY99UOWdlyLX9gHKgvHXr1jkuryNSd2WyG1Ky9+3CFH+HA9zBl7Q7xH3atKSur1pQLFdtlfjIxU5MwNbAq8A2QCPgaaB3ssE99qWqkCLh0t0NqbpqjiUl7s1Z5v+in6/D/Eu287MZ7SWt12d0z0Ld9SgfJQrumQyo9gA+c/dlAGb2H+BgYImZtXT3L82sJbA0g98QqffSWdhTVhYsJIqu9txoYdG6dfx2wX38jSFsybcMZyDXcx3fsSX2eZr3TLOtkhuZ5NwXAgeaWbGZGdAdmA08C/SJnNMHeCazJorUb4kW9oTl4hMu43/nHdh/f+7hQj5gT/ZmOpdzO9+xZcL7JlMaQIuQ8khYlz6ZF3A9MAeYCYwBmgDNCWbJzIu8N6vuPkrLiIQLy2NfeGF4fjveBhjbsMRHcV7wYYcd/I2LH/HipuuTvm/YRtOxm2oo516zyEXOPZsvBXeRxOLluhPlt2O/K+IXH8AdvpIt/WcauV95pfv336d836Ki5PLpmezcJKlJFNy1E5NIHRNNkcTbvQiClZ1jxgT58C6rXucu+rMnHzKxwVGsuvmfnPiXDgnv36BBELbjKS6unJopLtac9tqknZhECkR0UDMssEOQ3y49YjEz9y7ldX7DlnzLn7Z5iq9Gv1RtYI9eH090cZJKA9QNCu4idUi8Qc1YWzZdw38Ovg3at2fnqU/BNddQ8uNs7l16KqW9DQgfhI0eX7AgCN6xouUE6vvWdXWJ6rmL1CGJphT+fttXGNFwAFs+MgdOOAGGD4ddd610Tth0xrfegtGjNxx3DwK8e9BDL8TNpQudgrtIHdK69cYpmZ1YyD3Fl3P80idhl11g/Hg4/vi414dNZxw5Etatq3w8Gtjnz89e+6XmKC0jUofE1nRpzM8M5kZm05Fj1j4XbEz90UehgR3Ce/5VA3t150v+U3AXqWXV5cBjj0frnffZ9gVmsjs3MoTl+x1Lw7mz4ZprYJNNEv5W2GBpUUhpdi0+qsPC5kjW5Evz3KW+SnWB0rjbP3U/6aTgQLt27i+9lNPf0xz1/IYWMYnkVroLd5JdMLQJq/xa/s9X2ybum27qfsst7j//nFYbws7T4qO6R8FdJIcyWXIfr0xA5dd6P4mn/VPauIM/whnun3+echsUuAtTouCuFaoiGYrODa8qmZkmYdcWFcHO6+ZxB5dyHC8wk8705y7mlxwe956J2jBsWOXpj6CVpYVCK1RFciiTMrfxdjRq0fRHxu95NTPZnUN4i8u4nX2YxpTiw0P3JU3UhmSqOUrhUXAXyVAmZW6js19KSsBwLmzxBPObduDYaTfxRbcz6bFjBXfYZbQqaZSwp52oDaqxXj8puItkKF7v2yxIk7RoEbwS7SdaWgrzn5/F+iN7MOLr37Hg++Z0402O/Hw0A2/ePqml/mF7mg4bphrr9VZYMr4mXxpQlboutlRuokHSjQZav/3W/fLL3Rs29J+Kt/JLG93tDVib1nTERLNgNM2xMKEBVZGaETawGaukBOZ/5vDww/CXv8BXX8Ef/sC+L93EtEXbxD9/fmbtipYJXrgw6LGrVkxhSDSgqtoyIlmUTB57ywUfwG/6wxtvQNeu8PTTsP/+TA9JkmYjN15aqmBe3yjnLpJFifLYW/INd3AJ77MPzJoVjKS+9x7sv3/Ca5Ubl3QouItkUdzBVdZzLg9SQXsu5m4+6XEBzJ0L558fjLQmuDY6KCqSKgV3kSyqNLXR4Mgtp/Jew0N4kD+wqMmuvPS3ctpNuBuaNav2Wu10JJnQgKpILixfHoxgjhwJ224Lt94KvXtX6qmLZEorVEVyIG6p3nXr4N57oV07uP9+uOQSqKiAc85RYJcapf+2iaQhdqNq9+D9vr7vsrztAXDBBbD77jBtWrDV3ZZb/npNvLrtIrmg4C6Shth6LduwlAf4A5N+Poi1CxcH89cnTYI99vj1/Hj/GPTrpwAvuaPgLpKk2J73ggVQxFr6cycVtOdsxnALV9J2XQWcdVYwIhojrHhX797qxUtuaBGTSBKiPe9ogO7GG9xFf/biAybQgwHcSQUdKCmJf32ihUjRXjxoZoxkj3ruIkmI9ry350vG0Js3OIyt+IZTeYqjeZkKOiSck17dQiSV4JVsyyi4m9lWZvakmc0xs9lmdpCZNTOzCWY2L/K+dbYaK1JbFi/4hT/z/5hLO07nCW7gr3RkNuM4FTOrdk56vAVKVakEr2RTpmmZO4AX3f00M2sMFANXAxPd/WYzGwQMAq7K8HdEas/EicxsdAntfpnFeI5nIMP5hN2A5It6RYP+kCHhhcVUZkCyKe2eu5ltARwGPADg7mvc/RugJzA6ctpo4ORMGylSKz7/HH73O+jRg5Zbrea0xs9yIuN/DeyplgYoLQ3+IRg7VmUGJPcyScvsAiwDHjSzaWZ2v5ltCmzn7l8CRN63jXexmfUzs3IzK1+2bFkGzZD6JufzxX/+GW66CTp0gP/+F4YOZfOFszhl1IlZKQ2gMgNSE9IuP2BmXYF3gUPc/T0zuwP4Dhjg7lvFnLfS3RPm3VV+QJJVddYKZHmz5xdfDFaVzpsHJ58M//hH8C+ISB7KVfmBRcAid38v8vlJYF9giZm1jPxwS2BpBr8hUkk6mz0n1dOfPx9OOQV++9vg84svwrhxCuxSZ6Ud3N39K+BzM2sfOdQdmAU8C/SJHOsDPJNRC0VipLrZc7UrQ1evhqFDoWNHePnlIB3z4YdwzDE5ab9ITcl0tswAoCwyU+ZT4DyCfzAeN7O+wELg9Ax/Q+RXrVvHn20SNtMktKd/tVO6+X9h4ED47DM44wy47TbYccfsN1qkFmQ0z93dp7t7V3ff091PdveV7r7c3bu7e9vI+4psNVYk0YYW8dIv8Xr0uzGPEQuPh549+WZNU5g4ER59VIFdCopWqEqdEjbTBOKnX2L3xCjmR/7GEGayO914k8u4nZIV0yn78sjaeRiRHNJmHVIQ2rSJn65p3hxWr3J+u/opbufPtOZz/s3ZXMmtLGF7IPmFSCL5Rpt1SMELG1DdZvkcPt71aJ7kdFbQjG68QR/+/WtgT3StSF2m4C55JdUFStHzq/4f0M34nlu4kg/Yg00/msKUc+6kV+ty3qLbRveoOhgb24YWLYKXNtiQOsfda/3VpUsXFxk71r242D0I1cGruDg4nuz5sN7Posy/oKU7+H309W1Y4sXF7hdeWP39498zufaI1DSg3EPiaq0Hdldwl4iSkvgBtaQkufN35wOfxGHu4FPo4vvz7kb3GTs2eDdzb948eJlV/i4ssFfXHpGalii4a0BV8kaDBhunVyCYFbN+ffj5W/IN13MdF3M337AVg7mJUfyB9RSF3iesjEHVOfHxhLVHpKZpQFXqhLCFSGHHS3ZaTx8eooL2DOBO7uN82lPBg0XnbxTYq94nbHFT0caXJd0ekXyi4C55I9ECpY28/z6Tm3TjIc7jU3ahK+VcxD38VNycfv2qv0/YDJl16zba/jS59ojkGQV3yapMyvEmVQp3xQq48ELo2pVtvv2Ed/o9yO9bv8V02/fX80eMqP4+iXrf7hsCfPPmwUuleaXOCUvG1+RLA6qFIdXZLtFrogOc0UHNuNaudR85MhgBLSpyv/RS95Urs9pWDZxKXUOCAVX13CVrUi3HW23FxqjJk+HAA4MvO3eGadNg+HDYaqu4901G7P9LCKPFTVKXKbhL1qRajrfafwyWLYM//hEOOAAWL4aHH4ZJk2CPPbLS3ui2d2EBXgOnUpcpuEvWpDrbJSzof7FgLdx1F7RrB6NHwxVXwJw5cNZZiUc705TSQK5IHaHgLlmTapCMF/QP4U1mNOoKAwZA167wwQfw97/D5ptnv8ER2tNUCpGCu2RNqkEy9h+D7fmSf3M2b3IoJVusgCefDHZG6tixxto+f36wOGn+fAV2qfsU3CWrYoPksGFB/jxsWmRpKdw34heGbn07FbTndzzOzJ5D2HTBbOjVq1IKJpMpliL1Uabb7InEVXV5f3QmDMT0iv/3P35/a39YOSvYmPqOO9i9bdv07iUilajnLgml22MOmwnTuzccuOMiFhx4Bhx5ZLBB9TPPwHPPQZzAnuheYVMsRUTBXRJIeh56HPFmwjRiDVdxMxO/aM+27z3LjF7Xw0cfwUknxZ0FE/2HJd4OS2G/ISIBBXcJlUmPuepMmKN4mQ/Zg5sZzMscTSdm0bP8WmjaNO71sf+wJPsbIrKBgruESnVRUqzoTJgS5vMUp/Iyx2A4x/ICpzKO+eyc8D7x/mGJpXnoIokpuEuoVBclxSrt9RNv/3Yoc6wjx/ASg7mRPfiQlzg2qfskCvyahy5SPQV3CZXsoqRKg64lzqTL/wudO7PXU9exyWkn8vIdc/hn8WDW0CThfWKF1nAv0Tx0kaSEVRSryZeqQuav6qo2xlZX3JV5Pp7j3MG/2aGj+yuvJH2feL+baoVJkfoG7aEqUakG2eqUlLg35Ue/gSH+E439Wzb3y/h/vmvrNXnXVpFCkyi4aw/VeiRs39C089fu9Gowjtu5jBIWMobeXMmtfEVL7TMqUgNyuoeqmRWZ2TQzGx/53MzMJpjZvMj71pn+hmRHVhcDzZkDxxzDU/TiW7bkUF7nHMbwFS0BTVMUqW3ZGFC9FJgd83kQMNHd2wITI58lD2QytfFX338PV14Z1FSfPJkp59xJt6bv8yaH/nqKpimK1L6MgruZ7QgcD9wfc7gnMDry92jg5Ex+Q1IXVjIg0dTGassMuMMjj0CHDkEJ3nPOgblz2W90f+65r6HK5Yrkm7BkfDIv4EmgC3A4MD5y7Jsq56wMubYfUA6Ut27dOucDD/VFolkmYd9deGE1M1M+/ND9N78Jvth3X/d33slosFMDpSLZQS5mywAnACMif6cc3GNfmi2TPSUllYN01c2e4wXWsGt23+mbYCPqoiL3Zs3c//Uv97VrM5qmqCmOItmTq+B+E7AImA98BawCxgIVQMvIOS2BiurupeCevOp6vWbxA7VZ+D2rXmOs83N4yL9i2+DLP/3J/euvfz2/un9AEsnkWhGpLFFwTzvn7u6D3X1Hd28DnAm86u69gWeBPpHT+gDPpPsbUlkyVRrTKRkQ+93eTONNujGac1nceGeYMgX+9S9o3vzXczIZmM3KoK6IVCsX5QduBo4ys3nAUZHPkgXJTGVMZ7PnYcOgVdMV3M1FlNOV3fiYPzV+kFn3vw1dumx0fiY1ZzK5VkRSENalr8mX0jLJSTblktKA5bp17iNH+urNmvtaGvgdXOJ77LQy4TXKuYvkB1R+oDCkk69OGOjfe899v/2Cmxx6qPuMGQl/P/ZezZsHL82WEak9Cu4FItVeb9j5T4xY6v7HPwbRdfvtgxPXr8/qb4tI7iUK7qotU8eUlQU59oULgzz1sGHhC4aqblHXgHVcwL+4scFf2bLBDzBwIFxzDWyxRbW/G7bdXbQEr4jUvES1ZRTcC1iDBkEfG+Bg3uIu+rMP05nIkXT/6E7o1Cmte8VSgTCR2pPTwmGSG9WWA0hC69awPV8ymnN4i240Zzmn8QR9W7+SUmCP3iuV4yJSuxTc81Ay89mr9csvPNXtH1TQnjN4jBsZTEdm80LxaQy70X79nWT/AUlniqWI1KKwZHxNvjSgWlnGqzhffdW9c2d38C/2PNZ/s8PcjWampDNAqlkuIvkFDajWLWnntxctgiuugMceC7riw4fDSScFF1ahAVKRuk859zom5fz2mjVwyy1BOd5nnoH/+z+YNQt69owb2EFlAEQKnYJ7Lagu151SfnvCBNhzTxg0CHr0CIL6dddB06YJ25BRbXcRyX9h+ZqafNWnnHuyue5q89vz57ufempwg912c3/++ay0o9ra7iKSN9AK1dqRSu30pAdLV692HzrUvWnT4DVsmPtPP+VP+0SkxiQK7hpQzZHodMbYKo7FxRtXdYxKajHQ+PFw6aXw6adw+ulw221Zn2iuxUoidYcGVGtBWHneoqL45yeM0Z98AiecACeeCE2awCuvwOOP52QFkRYriRQGBfccCZt1sm5dCoOlq1bBtddC587w2mtBT336dOjePevtjdJiJZHCoOCeI2E93ZISGDkyeDfb8LlS8S93+M9/oGNHuOEG6NULKirg8suhceOctru0NIn2iUjeU849R8Jy7okCZVkZPHBlBVctvoRjeJmVO+3B1mPvgsMOq5lGi0idopx7LUi1B/zYAz+w5NyreHHxHhzIu1zCHZR8/T5lnyuwi0jq1HOvbe7w2GN81ftytl+3mAc5l0HczFK2A1QOQETCJeq5N6zpxkiMmTNhwACYNInF7MMpPMm7HFTpFJUDEJF0KC1TG779Fv78Z9h7b5gxA0aM4LTWUzYK7JD8FESVDBCRWAruNckdxowJCnwNHw59+8LcuXDhhdxwY1HaUxCzUv9dRAqKgntNmT4dDj0UzjknSKRPnuDE3FwAAAsZSURBVAz33gstWgCJB2DDeuXR4717x18wNWRITT6giOQTDajm2sqVwSbU99wDzZrBzTfDeecFkToJYVMq+/SB0aPDyxmASgaIFDoNqNaG9eth1CgYPBhWrICLLoKhQ2HrrVO6TVgZg5Ejg9WuiahkgEj9pbRMLkyZAgcdBOefH+TX338f7rwz5cAOicsYJKKSASL1m4J7Nn39dZBDOeCAICqPGQOvvw577ZX2LcN632EFyEAlA0Qkg+BuZjuZ2f/MbLaZfWRml0aONzOzCWY2L/Keene1rlm3Lsipt2sXpGIGDgxqwfTuHbrNXbLCCnn16xf/+NixwaInBXaR+i2Tnvta4HJ37wgcCFxsZp2AQcBEd28LTIx8Llxvvw1duwY59ei89dtvhy22yMrtw2bRjBihAl8iEi7t4O7uX7r7+5G/vwdmA62AnsDoyGmjgZMzbWS2ZHWhz5IlcO65cMghsGwZPPYYTJwYlOfNcptKS4Pe+Pr1lXvlYcdFRLIyW8bM2gD7AO8B27n7lxD8A2Bm22bjNzJVdUphdKEPpBgU166Fu+8O6qyvXh1sTD1kCGy2We21SUSkiowHVM1sM+ApYKC7f5fCdf3MrNzMypctW5ZpM6oVNqUwpYU+r70G++wDAwfy2pqDaP/LTNo8chNlz1Qf2OP10LPSJhGRODJaxGRmjYDxwEvufnvkWAVweKTX3hKY5O7tE92nJhYxZbQ36BdfwF/+Ao88wg8tSuj73XAeX9MTCAZLk6nTnvX9VEWk3stJPXczM+ABYHY0sEc8C/SJ/N0HeCbd38imsCmF7gny72vWwK23Qvv2wc5I11xD16azeHzNyUQDOwRBuk+f8Lx5VvdTFRFJhrun9QK6AQ58AEyPvI4DmhPMkpkXeW9W3b26dOniuTZ2rHtxsXsQzjd+FRcH5/zq5Zfd27cPvjzxRPdPPnF3d7Pwe4TdK9E1Vdu0UTtEREIA5R4Wo8O+qMlXTQR39yBolpSEB9qSEndfsMC9V6/gwK67uo8fn9S1ce8VEXZdScmG+5pt+CwikoxEwb3gV6jGDmQOGRIsCoq3rqgJP9F7wbCgXMDzzwcnzpwJxx9fqaRusmLLBoQtRBo2TNMZRSRHwqJ+Tb5y1XOPl4opLnZv3rzysd/ynM9j1+BDr15B7z1Goh57UVH1PfdoW9RDF5Fsoq733NNdfBQ2kAlBz3lnPuUZTuJ5jmctDTmKl2kx6Ula7Nu60m+FFe8yC8ruVu2VmwW9/Ni2qocuIjUqLOrX5CtRzz2s951MzzdsILOYH33GKdf6TzTx79nUr+QWb8TPoQOeVXv6VXvnsfn4qr+pAVIRyRXqcs89nYU+0Z6+bzSv3enJ01QUdWLPcUNp8vtedG9Vwa1cyS80jnuvVatg+fKN8/SxJXWjvfKSko1/U4uSRKQ25H1wD0uJhB0PG/xsy1xe5Fie5hQ2a7k5TJoEZWVMWdwqqXa4bwjwYUW6Um2riEiu5H1wD1vQE3a8ak9/U37gRgYzk905yN6lvPdwtvpsGvzmNwnvE497ENjDcuaptlVEJFfyPrgnmkYYz4ZesnM6jzObjgzmZh7m92zx5Vy6jrkUGm6olxbv/okk6oWn2lYRkVzJ++AeVs88bLZJ69bQiY+YSHce5wy+pgWH8Cb/V/IQbLddtfdv3jx4hUnUC0+1rSIiuZJR4bBsyVrhsO++Y/aZ17PbC//kezZnCMMYST82KS5KOciGFftSsBaRfJGTwmF5xT3Yr7RdOzq++A/mH3Ee3Xecy712ITuVpB7YQb1wEanb8ia4p71L0owZcNhhcM45QQR+7z3avjqSaZ+3yHjBkBYeiUhdlRfBfcWKDdMX3TfsSJQwwK9cCQMGwL77wpw5cP/98M47sN9+NdZuEZF8lRfB/YsvUliotH49jBoV1FgfMSLYmHruXMo26UubXRpkZ39UEZE6Li+C+5o18Y9vNO2wvBwOOgj69oV27WDqVLjzTsqe3zr1nr+ISAHLi+DeOP7K/w3TDr/+Gv70J9h//yBy//vf8MYbsPfegPYiFRGpKi+Ce6tW8Rf/3HjDOrjnnqCX/sADMHAgzJ0LZ59dqdiLlv2LiFSWF8G9WbONpx3+5y/v8Pvh+wc59b32CmbF3H47bLHFRtdr2b+ISGV5EdwhZtrhl0uYf8R5HHP9wbBkCTzyCLz6KnTuHHqtlv2LiFSWN8GdtWvhn/8MZsGUlcFVVwVTHM88M/6+eDG04EhEpLL8KD/QoYOXN24MH34IRx+9IciLiEioROUHGsY7WOMqKiKJ9v/AySdX21MXEZHE8iMt07IlzJoFp5ySUmBPu2SBiEiBy4+e+w47pFZUnY2rNkYXLoFy7SIi+dFzT4MWLomIhKuzwV0Ll0REwuVFcJ86NfWcuRYuiYiEy4vgDqkX+9LCJRGRcDkL7mZ2rJlVmNnHZjYomWuSyZlHZ8icfTY0bRrsd6qFSyIileVktoyZFQF3A0cBi4ApZvasu8+q7tpEOfOqM2SWLw9662PGKKiLiMTKVc99f+Bjd//U3dcAjwI9k7kwUc5cM2RERJKTq3nurYDPYz4vAg6IPcHM+gGRmelNgK6Ar1+wYOGCYB3TDq2gUWP4ZQ0s/gK+XgFdusT7sQULwGzq1Ow/Rk61AL6u7UbUMD1z/aBnrjklYV/kKrjHW2ZaqYiNu48ERgKYWbl7edz6CIUqeOb4NSEKlZ65ftAz54dcpWUWATvFfN4RWJyj3xIRkSpyFdynAG3NbGczawycCTybo98SEZEqcpKWcfe1ZtYfeAkoAka5+0cJLhmZi3bkOT1z/aBnrh/y7pnzop67iIhkV96sUBURkexRcBcRKUC1HtzTKVNQ15jZKDNbamYzY441M7MJZjYv8r51bbYxm8xsJzP7n5nNNrOPzOzSyPFCfuZNzGyymc2IPPP1keMF+8xRZlZkZtPMbHzkc0E/s5nNN7MPzWy6mZVHjuXdM9dqcI8pU/BboBNwlpl1qs025chDwLFVjg0CJrp7W2Bi5HOhWAtc7u4dgQOBiyP/uRbyM/8MHOnuewF7A8ea2YEU9jNHXQrMjvlcH575CHffO2Zue949c2333NMuU1CXuPvrwIoqh3sCoyN/jwZOrtFG5ZC7f+nu70f+/p7gf/itKOxndnf/IfKxUeTlFPAzA5jZjsDxwP0xhwv6mUPk3TPXdnCPV6agVS21paZt5+5fQhAMgW1ruT05YWZtgH2A9yjwZ46kJ6YDS4EJ7l7wzwwMB64E1sccK/RnduBlM5saKaMCefjMtb2HarVlCqTuMrPNgKeAge7+naWw+Xld5O7rgL3NbCtgnJntXtttyiUzOwFY6u5Tzezw2m5PDTrE3Reb2bbABDObU9sNiqe2e+71uUzBEjNrCRB5X1rL7ckqM2tEENjL3P0/kcMF/cxR7v4NMIlgnKWQn/kQ4CQzm0+QUj3SzMZS2M+Muy+OvC8FxhGkl/PumWs7uNfnMgXPAn0if/cBnqnFtmSVBV30B4DZ7n57zFeF/MzbRHrsmFlToAcwhwJ+Zncf7O47unsbgv/tvuruvSngZzazTc1s8+jfwNHATPLwmWt9haqZHUeQt4uWKSi4jfLM7BHgcIKyoEuA64CngceB1sBC4HR3rzroWieZWTfgDeBDNuRirybIuxfqM+9JMJBWRNBpetzdh5pZcwr0mWNF0jJXuPsJhfzMZrYLQW8dgrT2w+4+LB+fudaDu4iIZF9tp2VERCQHFNxFRAqQgruISAFScBcRKUAK7iIiBUjBXUSkACm4i4gUoP8PLQrD2Wh+PMIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def offset(_Y):\n",
    "    return _Y + np.random.randint(-9, 8 ,size=_Y.shape)\n",
    "\n",
    "def calc(_X, _W):\n",
    "    return _W[0] * _X + _W[1]\n",
    "\n",
    "X = np.linspace(0, 50, 100)\n",
    "W = np.array([2, 5])\n",
    "\n",
    "    \n",
    "base_Y = calc(X, W)\n",
    "off_Y = offset(base_Y)\n",
    "\n",
    "\n",
    "def draw_scatter():\n",
    "    plt.scatter(X, off_Y, color='blue')\n",
    "    \n",
    "def draw_line(_W, x_min=0, x_max=55, y_min=0, y_max=120):\n",
    "    plt.xlim(x_min, x_max)\n",
    "    plt.ylim(y_min, y_max)\n",
    "    _X = np.linspace(x_min, x_max, 100)\n",
    "    _Y = calc(_X, _W)\n",
    "    plt.plot(_X, _Y, color='red')\n",
    "    \n",
    "draw_line(W)\n",
    "draw_scatter()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAD8CAYAAACW/ATfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO2deZhU1bW330UzNo0CTYMIMkQJxmjEiJpcczFXrppIPjFXURQQ0UgSjaLGiWCuMYaE6xBDbsRcHBCxHUCNGmOciHFIjBEUjOLMPAh0I4My9LS/P3Zt6tTpc2qeumq9z9PPqTrjruqq3/nV2muvLcYYFEVRlNKmXaEboCiKouQeFXtFUZQyQMVeURSlDFCxVxRFKQNU7BVFUcoAFXtFUZQyIKHYi8jdIrJJRN72rLtJRN4TkbdE5A8i0t2zbaqIfCQi74vISblquKIoipI8yTj7e4Bv+dY9BxxqjPkK8AEwFUBEDgHGAl+OHDNLRCqy1lpFURQlLRKKvTHmJWCLb92zxpimyNN/AP0jj0cDDxpj9hhjVgAfAUdnsb2KoihKGrTPwjnOAx6KPO6HFX/H2si6VojIZGAyQNeuXY88+OCDs9AURQlm8eLFdcaYmnxft1evXmbQoEH5vqxSJqTyuc5I7EVkGtAE1LpVAbsF1mMwxswGZgMMHz7cLFq0KJOmKEpcRGRVIa47aNAg9LOt5IpUPtdpi72ITAS+A4w00QI7a4EDPLv1B9anew1FURQlO6SVeiki3wKuBk4xxuz0bHoCGCsinURkMDAE+GfmzVQURVEyIZnUyweAV4GhIrJWRM4Hfgd0A54TkSUi8nsAY8w7wHxgGfA0cJExpjlnrVeUDNC0YqWcSBjGMcacFbD6rjj7TwemZ9IoRckT92CNy72edc8BU40xTSLyP9i04qt9acX7A8+LyBfVzBQXjY2NrF27lt27dxe6KVmlc+fO9O/fnw4dOqR9jmxk4yhKm8QY85KIDPKte9bz9B/A6ZHHe9OKgRUi4tKKX81DU5UkWbt2Ld26dWPQoEGIBOWLtD2MMdTX17N27VoGDx6c9nm0XIKihHMe8OfI437AGs+2uGnFIrJIRBZt3rw5x01UvOzevZvq6uqSEXoAEaG6ujrjXysq9ooSQKZpxcaY4caY4TU1eU/tL3tKSegd2XhNGsZRFB8lk1b8xhvQ3AxHHVXolihFgDp7RfFQUmnFU6fCFVcUuhVlR1VVVaGbEIg6e6VsiaQVfxPoJSJrgeuw2TedsGnFAP8wxvzAGPOOiLi04ibaQlpxQwM0Nha6FUqRoM5eKVuMMWcZY/oaYzoYY/obY+4yxhxkjDnAGDMs8vcDz/7TjTEHGmOGGmP+HO/cRUFLi/1TCoIxhiuvvJJDDz2Uww47jIcesiXENmzYwIgRIxg2bBiHHnooL7/8Ms3NzZx77rl797311luz3h519kpp09AAN99c6FYUhnIX+0svhSVLsnvOYcPgN79JatdHH32UJUuWsHTpUurq6jjqqKMYMWIE999/PyeddBLTpk2jubmZnTt3smTJEtatW8fbb9vxfVu3bs1uu1GxV0qdq66CmTML3YrCYEx5i32BeeWVVzjrrLOoqKigT58+HHfccbz++uscddRRnHfeeTQ2NnLqqacybNgwvvCFL7B8+XIuvvhiRo0axYknnpj19qjYK6XLI49YoZ8ypTwFv9ydfZIOPFdEE7liGTFiBC+99BJ/+tOfmDBhAldeeSXnnHMOS5cu5ZlnnuG2225j/vz53H333Vltj8bslaKlthYGDYJ27eyytjbRER4++gjOOw+OPhpuvDFHLSxyWlqsu1cKwogRI3jooYdobm5m8+bNvPTSSxx99NGsWrWK3r17c8EFF3D++efzxhtvUFdXR0tLC6eddho33HADb7zxRtbbo85eKUpqa2HyZNgZSX5ctco+Bxg3LsHBu3bBmDFQUQHz50PHjjlta9FS7s6+wHz3u9/l1Vdf5fDDD0dEuPHGG9lvv/2YO3cuN910Ex06dKCqqop7772XdevWMWnSJFoi/69f/epXWW+PhP3UyCc6eYniZ9AgK/B+Bg6ElSsTHDx5MtxxBzz5JIwaBYCILDbGDM92OxNR0M/28OGwZw/861+FuX4BePfdd/nSl75U6GbkhKDXlsrnWsM4SlHhQjdBQg+wenWCE8ybZ4V+6tS9Ql+2qLNXPGgYRyka/KGbIAYMiHOCZcvgBz+AESPg5z/PevvaHJqNo3hQZ68UDdOmxRf6ykqYHjZTwmefwemnQ1UVPPAAtFcfU67OvhhC09kmG69JxV4pGuKFaAYOhNmzQzpnjYEf/hDeew/uvx/23z9nbWxTJCv2e/bYgmklQOfOnamvry8pwXf17Dt37pzRedT+KEXDgAFpdsrecQfcd58N3YwcmavmtT2SFfvDDrM3y8suy32bckz//v1Zu3YtpTaPgJupKhNU7JWiYfr01jH7uKEbgDffhEsugRNPtHEgJUqyYr92rf0rATp06JDRbE6ljIZxlKJh3Dgbqqmujq7r0iXOAdu22Xz6Xr2ss2+nH+cYkhX7Mo3tlxvq7JWiY9eu6OP6+pDBVMbA+efb+M6LL4LOCNWaZLNxVOzLArVCSlERlJGzc2dAhOa3v7W1b2bMgGOPzVv72hTJlktQsS8LVOyVoiIsIydm/Wuv2RmYTjkFfvzjvLSrTaJhHMWDir1SVIQNmtq7vr7exun794d77oESnFw6ayQj4sbo4KsyQWP2StFQW2vHRvnZm5HT0gLnnAMbN8Irr0CPHnlvY5siWbF3+yoljYq9UhSElUqorral6MeNA2bcCE89BbfdBkcdVZB2timSEXu3XcW+5NEwjlIUhJVKqKqKCP1LL9mdzjzTDgBSEpNMeEbFvmxIKPYicreIbBKRtz3reorIcyLyYWTZw7Ntqoh8JCLvi8hJuWq4UlrE7ZjduBHGjoWDDrKjZTVOnxzq7BUPyTj7e4Bv+dZdAyw0xgwBFkaeIyKHAGOBL0eOmSUiFVlrrVKyhHXMDjqgGc4+Gz79FB5+GLp1y2/D2jIq9oqHhGJvjHkJ2OJbPRqYG3k8FzjVs/5BY8weY8wK4CPg6Cy1VSlhpk+3HbFeKivh8a9eD3/5C8yaZWu4KMmjYq94SDdm38cYswEgsuwdWd8PWOPZb21knaLExZVKGDjQRmkGDoQnLn6Wwx7/BZx7LkyaVOgmtj1U7BUP2e6gDQqmBg7hE5HJIrJIRBaVWoU6JT3GjbPVD1paYOUraxl51zg49FC47bbMJh8PoeT7o1IR+xIqCawEk67YbxSRvgCR5abI+rXAAZ79+gPrg05gjJltjBlujBleo3VNFC+NjTbrZvduWLCA2j9UMnmyLX9sTHTy8SwI/j2Ucn+UGzAVD3X2ZUO6Yv8EMDHyeCLwuGf9WBHpJCKDgSHAPzNrolJ2/OQn8Pe/w513wtChydfLSZGS74/yOvs774SlS4P38S6VkiWZ1MsHgFeBoSKyVkTOB2YAJ4jIh8AJkecYY94B5gPLgKeBi4wxpTEFjpIfHn8cbr4ZLrzQunuSrJeTPUqnP8or9pdfbstLBO3jXSolS8IRtMaYs0I2BU4JZIyZDsSbbkIpY2prrSNfvdqmW7qJSaZNg3arVvBmu3NpGnwk1b/+9d5jwmawijv5ePZJqT8KmAwwIM+NjMEr9s3NwVMPqtiXDTqCVskbriSCN/Y+aRKcdx58smo38xmDaTH8+4YF1D7cae9xYWmZcWewSp/S6Y/ydr6Gddaq2JcNKvZKXqithYkTW8feGxuhoQFu4ccMZzETmcu7uwfvjce7XwI7d0JFpDs07uTjmVM6/VFeIVexL3u0EJqSc5yjD4oiAJzJg1zELG7mxzzBaMCGefzF0Zqbo44+G0If6Y/6JtBLRNYC12H7n+ZH+qZWA2PA9keJiOuPaqIt9Ed5K1o2N6vYlzkq9krOmTIluMgZwFDe406+xyscy1R+tXf9gAHxZ63KhtiXfH+UOnvFg4ZxlJxSW2vnGwmiCzt5mDHsogtjeZAmOgBR957nLJzSw9s5a4x20JY5KvZKTomXC3+b/Igvyzu8dXUt7Qf231smwcXjE85apYTjHUzlRF6dfVmjYq/klDAXfi5zmGTmINdey8gZJ0bLJKyMhmjynIVTWnjFu6mp9Tr/fir2JY+KvZJTglz4YbzFLC6E44+H664LPTaoOFoOs3BKCxV7xYeKvZI1goqVnXxy7Fwj3djOwzIG070H3H9/NJ8yhJjiaCtV6JPGG8ZRsVfQbBwlS/jTJN2AKRGv7hhmM5mD+Ih2j/0F+vQpVHNLH694NzbapXbQljXq7JWsEJRe6QZMOX7I7YzlIW7adzocd1x+G1huJArjbNpkp3tUsS8bVOyVjImXXuk4kkXcymX8iZP5ydar8tOwciaR2J9/PlxwgYp9GaFiryRFvMlDpkyJf2x3PmUBY/iE/TiHe+lR3W7vuXr1sn/ZnJREIbHYb9kCW7fq5CVlhMbslYQExeMnT45uj+fqO3Yw3NN4Lv1Yx7/zMjs6VCM7osd4j/WeVztiMyQoZu9d58+9V2df8qizVxISr2xBvEFT1dXw6um3MJonuJob2TjwGPbZJzaO7ycbk5IoBGfjeDtom5piSyio2Jc8KvZKQuKVLYhXumBo/d/4ygPXsPqo07i1ZQorV9roQbrXU1IgURjH1bdXsS8bVOyVhMQrWxC2rRebeYgzWcFgjnn7Lmrvl7jnSuZ6SgokEnt19mWHir2SkHhlC4K2taOZ+xhPL+oYwwI+2bXv3tDM9Omxg6z8aDmELJGJ2L/9Nixblvs2KnlFO2iVhLjOUv90gt5OVLfNGPgJv+QknuUCZrOUYUA0NDNuHPztb/D737dOAKmuhpkztXM2K2QSxrnkEujYEZ5+OvftVPKGOnslKeKVLfBuO6v3Qq7nOuYxnjv53t59vKGZWbNg3rzYmjf33Qd1dSr0WSNI7JPtoN21y/4pJYU6eyV7rF/PXbvP5n05mB+a23FzdAeFZsaNU2HPKYlq48Rz9i0t0WOUkkGdvZIdmprgrLPo0vQZH814mM7VVXs3delSwHaVK4ny7OM5ezeNoVJSqLNXssNPfwovvQT33sv2dofERAHq63WwVN7JpIPWzWyllBTq7JW08JZPmNT7TzBjhq21MmFC3EFYSp7IpINWwzgliTp7JWW85RMGsIpbNk9gqQxj2dd/y1no3LFFQaISx4mcvVJyqLNXUqK2FiZOtELfgQbmcwYVNHOaWcDU6zsDOndsUaDOXvGhYq8kjXP0zvjdyFUcwz85j7v5mIP2OnedO7YISJSNk8jZq9iXHCr2StJ4Y/Gn8TCXMpOZXMKjnAZEnbvOHVsEJNNBG8/Zayin5MhI7EXkMhF5R0TeFpEHRKSziPQUkedE5MPIske2GqsUFufcD+Qj7uY8/sExXMlNQGvnrnPHZpnt2+HnP0/ecScTxvE6e+8vAQ3jlCRpi72I9AMuAYYbYw4FKoCxwDXAQmPMEGBh5LlSAgwYAJ3ZxQLG0ER7zuQhGulIRUXpOfeiMzLPPw/XXQdvvZXc/vHEvqXFiruGccqKTMM47YEuItIeqATWA6OBuZHtc4FTM7yGkkXizTiVaJ/p0+G2iikcwRImMI/VDETEasO0aaUzy1RRGpk9e2KXiYhXLsG71A7asiHt1EtjzDoRuRlYDewCnjXGPCsifYwxGyL7bBCR3kHHi8hkYDLAAE3TyAvxZpxyrjzuPi3zoPkOZu1zDU9tH4VI9Nd/Cc4y5YxMI1EjMxX4ZmT7XOCvwNV5aY2b8SUTsXfrvM/DnL3G7EuOTMI4PbAufjCwP9BVRMYne7wxZrYxZrgxZnhNTU26zVBSIGyw05QpUSfv0ir9+9xz5Ts0XfADXus0ginbb6CiovUgy1IZOGWMWQc4I7MB2GaMeRaIMTJAqJERkUUismjz5s3ZaVSqYu/95/jLJXidvjr7siGTMM5/AiuMMZuNMY3Ao8C/ARtFpC9AZLkp82Yq2SBsUFN9vXXmxgQbuq58xswNY9iyp4rv7nmAJtqHGr9SGDhVlEYmm87eO/+sxuzLhkzEfjXwNRGpFBEBRgLvAk8AEyP7TAQez6yJSrZIL1pmuJ0fcjDvMZYH2MD+ObhG0VF8RsaJfbwJfL1kEsbR1MuSJG2xN8a8BjwMvAH8K3Ku2cAM4AQR+RA4IfJcKQKCBjsl4nvcyQTu42f8jBc4Pu6+JTRwqviMjAvFpOPs/eUStIO2LMmoNo4x5jrgOt/qPdgvh1JkuI7TiRMTG7eKCjis+U3+l4t5hhP5BdeG7tfSEjx7VVvFGPOaiDgj0wS8iTUyVcB8ETkfe0MYk7dGFaKD1pj4c0gqbQothFZmjBsH4xNEnysrYc5vtvHVyWOooxfjuQ8T8COwsrL08usdRWdkciH28Zy9W1ZUpNdepejQcgllRm1tfLM2cCDM/j/DGc+czyBWciYPUUfrTkYtgZBnMhF7b4es93nQNu86DeWUFOrsy4xp04LnpRCx88ICfHTxb+HTR/hpx5v5e8OxMfuVspsvalLtoE0m9TJom3+/Tp1Sb6tSlKjYlxlhqZFOG+44/x88t+cKHmM0Mxouj9lHxMb7VegLQDbCOP4OWu+2IGevGTklhYZxyoyw1MiBA+Gma+q5d88ZrKU/k5iDmzDcYQw89VTu26gEkIuYvfexMdE7ftB+SptHxb7MCK01f0ML09eeQx82MoYFbCW4xlcpDJpqk+RK7F0YB1TsSxwV+xLGW9CsVy/7N2ECdOkC1dW+WvPrbmQUT3EZt7KY4aHnLJFBU22PTAZV+ePyQWEct93r8DWMU1JozL5E8Rc0q6+Pbquvt25+3rxI/P3FF+Gcaaz82ljmLv2hLWsXQAkNmmp75MPZt7TEpmqpsy8p1Nm3ccLKEQcVPfPiipY9Mmsjm0aO5f2Wgxi1bjYTz5W9M0xVVwf8AtDO2cKQSSG0ZDpo3TFh25Q2jzr7Nky8csTJxNbXrGqm+uKzqWrZxkieZdmabrz7e/udHziwdEbElgSZOPt4qZf+MI7X2WsYp6RQZ9+GCStZPG1acrH16+V6vtnyFy5kFm9zGNC6Pn2pTEjS5slFITRoHcZRZ1+yqNi3YcLc++rViYuendLpGX5ifsHdTGIu5wbuUyr16UuCXJQ49m5z24OOU0oCFfs2TJh7HzDAhl9mzyYw/n5Mv7U81HE8H3T4Mj/id3GvoamWRUImVS/9JRHiOXv/cd//PjzzTOrtVYoOFfs2TJh7X7XKplkCrFxps26qqmDLFvjCAY080XUsnc1uPpj+MJKg5rGmWhYJ2YjZu/XxnL1/291328nOlTaPdtC2YVzn6ZQpsamVYJ+fdx787W8wd240tv/91T+hN3/jlR89yClXDmX2/jZUs2oVMXPKgqZaFhXZyMaB1rXq/TcCf02dpibtqC0R1Nm3ccaNs649iIYGG8pxQn8Kj3MlN/M7LmL8H8/ce/zKlfY7Pm9eNOyjqZZFRjY6aN36eNk4XmHfvdsuVexLAnX2JUC8uLr7ng5mOXOZyCKO5MfcQmPAMePGqbgXLdnooHXrk+2g3RUZXadiXxKosy8B4sXVKyqgE7uZzxkYhDEsoIFOGotva2QzZp9sB62KfUmhYl8CTJ8OHTq0Xt+xo82Vn9n+xwxnMedyDysZrLH4tki2nL2bcjBom9/1O7HXFMySQMW+BBg3DubMsamVjupqm0gxa8SDfL9pFrP3+TF/lNEaiy9WbroJrrwyfHuuYvbxnL3r7FFnXxJozL5ECIy3v/8+DL8Ajj2WyS/8iskB7l8pEv76V1i3Lnx7Jtk48cI4yTh7FfuSQJ19GyasCBpgXdnpp0PnzvDgg8FxHqV4aGoKD5cYk7sOWo3Zlw3q7Nso8YqgAXT8/kWc9vk7nNv7z5z0Yn8N2xQ78fLZnViLWLE3Jv6s8ZCd1EsV+5JCxb6NElYEbcoUOH3HHH7fcA/X89/M23QSj0RuAir4RUxzc7izd66+qgp27LD7JfqlFlQuwT3W1MuyRMM4bZRVq4LX71//Frc2XMhCjufn/DcQvQkoRUw8Z+8Ve+/zeKSTemmMin0Jo2LfBqmtDf4VX8UOFjCGT+nB2dxPCxV7t9XXa7niVBCR7iLysIi8JyLvisjXRaSniDwnIh9GlsET9aZDvJi9E/du3ewymbh9NsI47qejpl6WBCr2bZBp02KTLSyGO7iAg/iIs3iATfQJPE5JmpnA08aYg4HDgXeBa4CFxpghwMLI8+yQjLNPRezj1cZJtoNWyyWUFBmJfd7djwIEl0f4Ibczlod464zpvMRxSR+ntEZE9gFGAHcBGGMajDFbgdHA3Mhuc4FTs3bReDF7J8jpOnv/eu2gLUsydfb5dT8K0Lo8wpEs4lYu4y9dTuaIB66KGVwV7zgllC8Am4E5IvKmiNwpIl2BPsaYDQCRZe+gg0VksogsEpFFmzdvTu6KqTj7VGP2Xvw3lXgdtDqoqqRIW+wL4n4UILaOfXc+ZQFj2Cj7Uf/re6FdO2bObF3nXkskpER74KvA7caYI4DPScG0GGNmG2OGG2OG19TUJHdQMjF710GbqbOPF8ZRZ1+yZOLs8+9+SpC4A6NC2DsL1QDDPZxLP9ax7Lr5jPlBdex2LVecLmuBtcaY1yLPH8aK/0YR6QsQWW7K2hWzHbNPN4yj2TglSyZin3/3U2K4gVGrVtn+tFQm+R43DlZefAujeYKOM2/mW9cd03r7SvvdXblShT4VjDGfAGtEZGhk1UhgGfAEMDGybiLweNYumkyefbbEXkfQliWZiH3+3U+JEW9gVEK3/7e/wTXXwGmnwcUX56G1ZcfFQK2IvAUMA34JzABOEJEPgRMiz7NDtlMvW6drWbSDtmxJewStMeYTEVkjIkONMe8TdT/LsK5nBtl2PyVGWHZMfX10mkFvGYS97nzzZjjzTHsnuOuuxEPnlZQxxiwBhgdsGpmTC+ZyUJV/far17DXPviTINBsnv+6nxEg2O2bnTk+OfHMzjB8PdXWwYAHsu2/O2qfkkaam1iNYHZl00LbzfcVTqWev2TglRUa1cfLufkqM6dNji5nFY9UqG84Zt/yX8Oyztsf1iCNy30glPzhBbW5uLdCZxOwrKmJvIKl00OqgqpJCR9AWEJc1E5YX7+f+8xfSct3PrLP/3vdy2jYlzzjRDQqZZCr2/vVe8fbfCDRmX7Ko2BeYceOiv87j0Zf13L3nbD5uPxRuv13j9KWGE/kgYc0kZt++fev1YTF4v7N3qNiXBCr2RUCiMgYVNPEAZ1HFZ5za+HBydwelbZFtZ++ycYKcvYp9WaJin0fCBlD17Bn/uBv4KcfxEj/g97wrhyBijX2vXlrJsmTwxuz9pBvGadcuvIO2Y8fgY4Ku71/X0gLr1ydug1JUqNjnibABVBdeCNu3hx93Mn9iKjOYzQXcx4SY9On6ejjvPBX8No8xUUHNZsw+SOydsw8T+yBn72/Tk09at1JXl7gdStGgYp8nwgZQzZ4dm+rsZQCrmMcE3mQYU5gZuE9Dg5YubvP4Z5Lyk27MXiRY7MOcvfemE9Y+gI0b7Yd269bE7VCKBhX7PBEWlw8Lh3aggfmcQQXNjGEBu+mS8rmVNkJYKqTDuYEuXWwMPtvO3u2TbMw+Xv+CUrSo2OeJsAFU/v4zx41cxTH8k/O4m485KK1zK22EZJ19+/bQqVP2xd59CJMVe3fzUbFvU6jY5wlvWWJHZaWN2/vX/xePcCkz+Q1TeJTT4p63Y0ctXdzmSeTsGxrsP1okebE3JvkOWpeemWwHrTr7NomKfZYJy7gJKzs8a1bs+qO6f8g9Mol/cAxT290Y91rV1XD33VrRss3jFc0wZ+/EORfO3iv2ztl7t6vYlwQZlUtQYnEZN64j1lvEDGxH6urVNuwyfXpUpMeNizzetQu+PgbWdOBrb87nzpc7MmFCcAHDgQNt6WKlBEjW2UPqYu8ffBfUQRvk7Dt2jIaPNIxTEqjYZ5F4JYt37Qq+CcS48ilTYOlSm9o2YADjxtnKCEFop2wJkUzMPh2xD8vGSdXZ+0VdnX2bRMM4WSReyeKgm0BMyuS8eXDHHbZG/ahRe1cPHBh8Tu2ULSFy6eyd2HfoEF0fT+zdzaZTp+h2DeOUBCr2WSRVAd57c3jnHfjBD2DECLjhhph9wjp2tVO2hEhG7J1YZyr2iTponbN3+7tjvGgYp02iYp9FwoQ5rKrlgAHAZ5/BmDF2wMyDD7YqXKXzyZYBueig9WfjeAVdO2jLEo3ZZxEnwP6OWGhdt76yEqb/wlhH/9578Nxz0Ldv6HlV3EuYsMlEHNkI4/hDNamEcbzn87ZRxb5Noc4+y7iJvufNs88nTLDiP3FirDufOBHennIH1NZyyz7XU/uJzvdStiRy9rt3R8U3WzF7r5gncvb+doWFcX72M7j11sRtUwqCin0OCCp6NneudfktLXa5ZM6bXLflEp7mJK7cNo3Jk7WgWdmSKGa/dSv06GEfp5uN4w/jeGPyYamXXoJ+ffjb+sc/wp//nLhtSkFQsc+AsAFU8VIwAWZM3ca9u8dQRy8mMA9Du9bZOUr5kMjZ19dH62BnGsZxHbRhYh/m7INuSH6xb2hIrkibUhA0Zp8m8QZQxUvBrL3P8LM15zGIlRzHi9RRs3e75s6XKYli9lu2ZC72/jBO+/Z2W0tL6s4+LIyjYl/UqLNPkzD3Pm1a/BTMZd+fyWk8ytX8D3/n2JhtmjtfpsRz9s3NNozjUrpSzcZxI2i9Yt/cHBV7SD1mH+bsGxvD63UrBUfFPk3CXPjq1eE58MfwD36280oeYzS/5vKYbSKaO1+2xIvZb9tmhTtbYRyvs3fVLt02Y1ITe7+wq7MvalTs0yTMhQ8YYDNy/Ln1PalnPmewhgOYxBwgtmaJMZpeWbbEc/ZbtthltsW+oiI8LROiYu+WyYRxGhtV7IsYFfs0CRpA1aGDHSPlvkPueyK0cC/n0IeNTN53Plvp0ep8YWURlMIhIhUi8qaIPBl53lNEnhORDyPL1v/IdIgXs89E7L3ZOP4RtInCOG7/Ll3C2xgUs9cwTtGiYp8m/pGt1R4ghewAACAASURBVNV2WV9vXbpbVlfDNfwPo3iKpRNvZdJtw7X8QdthCvCu5/k1wEJjzBBgYeR55qTq7J1gxyNRB21FReswjnP27dpF9+/cuXW7NBunTaJinwTxatSvXGm/I1VVrT/njY0wsv2L/LLdtXDmmRw954da/qCNICL9gVHAnZ7Vo4G5kcdzgVOzcrF4Mfv6erv0dtBCYnefaASt19n7Z6pq1y66zjl7b7s0jNMm0dTLBMRLsfQKdFCHbR8+4Tcbx8LQIbaiZSQzQssftAl+A1wFdPOs62OM2QBgjNkgIr2DDhSRycBkgAHJpFil6uzBir3/J6KXRLVxgjpo3Y2goiK6Lllnb4xm4xQ5GTv7vMU1C0S8FEsv/u90O5q5n7PpLttgwQLo1g2lbSAi3wE2GWMWp3O8MWa2MWa4MWZ4TU1N4gOSidl3726X6Tp7F5ZparLCHNZB645z65KN2bvH6uyLlmyEcfIT1ywQ8VIsvfg7bK/jeo7nBZZcMAsOOyx3DVRywbHAKSKyEngQOF5E7gM2ikhfgMhyU1aulsjZ77tvVHxTEfugcgneycvjib0/jJNI7N15VeyLlozEPq9xzTzj4vRBUwKCXR82x+xJPMO1/IKPR0zi6/93bp5arGQLY8xUY0x/Y8wgYCzwF2PMeOAJYGJkt4nA41m5YLyYvXf0LGTu7J0Yx+ug9YZxgsQ+KGbv1jU2hn9plIKSqbN3cc0Wz7qYuCYQGtcUkUUismjz5s0ZNiO7eAuZxcPF72tr7d+0adC8ai33txvHtgMO5cA//27v+YI6eJU2xwzgBBH5EDgh8jxzcin2/hG0TpQTOftUY/buJmJM4kwhpSCk3UHrjWuKyDdTPd4YMxuYDTB8+PCisgJBcfowvHPMNuxs5K+cSYeWPfz75gVc+Qcb10mmg1cpTowxfwX+GnlcD2S/FnW8OWi3bIkdoZdpNo43jBPP2acbxnGP22vuR7GRibPPb1wzx3jddyJH78fNMfsrpnIsf+d73MnS3UOZNi35Dl6ljEmUepmOsw/LxvGGcVJx9olSL71ZOJqRU5SkLfZ5j2vmEH/9+TCc2QliNI9xBbdwGxcynzMB24mbbAevUsYk6qD1ir0T33SdfaIwTrqpl35nrxQduRhUlZu4Zg5JJmxTWWlvCEGjX7/afTn3cC6LOJLL+fXebQMGxK+hoyhAuLNvaYFPPw129rt3xz9nWLmEoA7aZAZVJRJ7r5tXsS9KsiL2xpi/GmO+E3lcb4wZaYwZElluycY1ckk8l+0d5TprVuvRr3f+bjfPdh+DQRjDAhqwX0ZXAiFsEnItj6DsJSxmv327Fd9sZOPES72sqLAfaCf26WTjeAVewzhFiZZLINxlDxwYO5fsoEH2sSuRsHIlHFl7OdUr32Aic1lTMXjvca4EgpZHUBIS5uy3b7fLffeNrktX7N3joA5at81bGyfVQVUaxil6VOwJd98nn9x6LlnvXLGvXPQAX1x4OzdxBX/kFJqbo67dK+beGjorV6rQKz6caHboECuqu3bZpRNcSF/sXYdsUAetV+z9YZx4MfuwTlkV++zw7/8Ov/991k6nYk+4+37qqTiZNO+9xxG3X8ArHMtP+GXr7YqSLE48O3WKdcsuLu8E1+0DqWfjuMfuBtKpU+ttLkdewzjFweuvw1tvZe10mgwbIag42YQJwftuXrUTxoxhp+nCWB6kiQ4x2zXTRkkJb+jEK6qZiH1YGMcr9kFhnGRSL7WDNve0tNj/sft/ZYGydvaJRraGxfLv6XoRvPMOl/euZR39W23XTBslJVwVyvbts+fs/dk4LjTjfqoGOXtvB22mg6qUzHD/XxX7zPHn1vvj8RAcyz+Puxnz+T3M3Odaup12ombaKJnjFftsO3tXLsHF6BM5e38HrT9m7y2HoGGc3OH+Tyr2mZPMyFZvLB/gK7zF77iIhRzP5duuY+5cmDhRM22UDPHOHJXI2bt8+WyEcbzb/KmXztHvs49dBgm8hnFyRw7Evmxj9smObHWx/EMHbGfBmtPZSnfO5n5aqGDnTtuJu3JlzpurlDJu5qhknL1IcvPQpir2fmd/6qnwhz/AF74QbSOEi72GcbKLOvvskdLIVmP46ZoLOJCPOZOH2ESfvZu0M1bJGO/MUYmcPSQn9v5snGTDOO4m0aWLFXy3jxN7r4PX2ji5w/3vVezTw9sh+9ln0LFj7PbQePusWZzJfKYxnZcZEbOpXTstWaykyfvvw8yZiTtovXn2kL6zT6aD1qVeOvxiny1n39SUVSErOdTZp4+/Q7a+3i6rqxPE219/HS67jHWHn8zvulzV6rzNza07dhUlKe6/Hy691Aqwi9knCuNA8mLvz8ZJxdk7ciX2118P3/hG4v3isWdP7n9FLF8O//pXbq8RhPs/JaqBlAIlL/bOzY8f37pDtrERqqrijGz99FM44wzYbz/6LbyX/7ujXWDlSx1IpaSF+yJv35586iVkFrN3ZV0TpV463OOgUbOZhHFWrEi9lrifb38brrgis3Mk4uqr4dxzc3uNINTZp0YyM06FxtyNsf/kdevshOHV1YwbZ78LKZ1HUcJwYr5tWzRmH1QuwaVbOtIRe295BHeOeB20jlw5+927M3ety5fnPjti+3bYujW31whCY/apkUzp4tABULfcAk88ATfdBMcck3B/HUilpIzX2bsSBX5n36lTNFfeESb23/gG/Pa39nGYs/eeIyyMk07MPtXUy127Mhf7XbtyH/ffvTv5aeuyiTr71EjktkM7ZP/2N7jmGjjtNLjkkphNWrJYyRpOsF0YJyhm7w/hQLDY79ljP7eLF9vnYbVxHB07Jufs3eCq5maoq4uKukhmg6p277bn9M/MlQo7d+ZeiAst9nv2hIcTUqSkxT6e2w7tkN282cbpBw+Gu+5q5aq0ZLGSNfxhnCBnn6zYr1tnl/X1dhk0gta59I4d7fpUOmhXrYL99oPnn7fPO3duLfb+CVLikWkHpDH5cfa7duVP7N98M/qeel9XljppS1rsw1z4fffZbdOm+eriNDdb1a6vt3F6bx1xD1qyWMkKiWL2qYj92rV26RX7oGwcd7xb590WL/Vy7Vq7bcUK+9wv9o2N0S9bsjF7SNz3EEZjo21PPsI4TU25z/pZuxaOPBIeeyx6XUeWXmNJi32YC4fgujhLz5wOzz1n457DhhW28Urp44TOlUvIxNk7sd8SmRguXsw+ntiHOfsdO+xy2za77NKldZy+Y0fr7pMRxkydfQ5i2oG49uXa3W/YYMWors4+976uLL3Gki+XEFS6eNCg1v+7r+1cyGGP/MzmaF5wQb6ap5QzXqHLhbNPJPapdNB+9pldesXePQYr8E7sU3H26Yq9+wLnS+w//zz0l35W+PRTuwx6Xers08ffcduX9dzP2bzHwXD77a2zHxQlF/jFPhNnv2aNXX76aaxDD0q9DHP2rqJlPGfvpkoMitl37Gj/8hGzLzVn736RqdhnF2/HbQVNPMhYqviMS/o+DFVVCevcK6WNiBwgIi+IyLsi8o6ITIms7ykiz4nIh5Flj4wu5BXssEJoqTr7lhbruIOycZxwp+LsReyf39mHddAmK/ZtzdnnWuydsw+6CarYp4+34/YGfsoIXuaSjv/HpJsOSarOvVLyNAE/NsZ8CfgacJGIHAJcAyw0xgwBFkaep4/3Cx1W4thfFwfiiz3YUE66MXu/s3dtc2LvnH2XLq07aAsRs3cdtbmguTn6WvIl9kE3Mc3GSR/XcXtuzZ+Yygzu7XQBj3cbz4QJcM45ievcK6WNMWaDMeaNyOMdwLtAP2A0MDey21zg1IwulMjZ79qVWhinb1/72Il9Otk4fmfv2uYX+0ycfUtLdJ9MnT3kzt173+NCir06+8wY941VzGmawJaBw7i03W/3FkbTcgiKFxEZBBwBvAb0McZsAHtDAHqHHDNZRBaJyKLNmzeHnzyogzaZmH3nznabq3PT0AAbN8Lhh9vnW7YkF7OPN1OVF2+1TH8Yx7XBOftkxN4ropk6e//jbOJtW77DOLt2RfsOVewzoKHBDpxqauK7jQv4dFfAF8qHlkMoP0SkCngEuNQYsz3Z44wxs40xw40xw2tqasJ3DOqgTSZmP3CgFVdX9Gn9ert0Yp9OGMc7U1WQ2Ducw3fhJeeOUkm9TCVEccMNMGdO6/Ve8c1UiN98ExYubL3e2858d9Du3h3N/lGxz4CrroJ//hPmzOHlDQcldYiWQygvRKQDVuhrjTGPRlZvFJG+ke19gU0ZXcTrcMNi9kFif8QRdvnmm3bpMnGCxN65w3RTL737eXHtcu1NJYzjFfhEYn/HHXaAo59sOvsbboAf/aj1+kI4e28Yp2fP6OMsUH5i/8gjdsKIKVPgtNOScuzV1TpKtpwQEQHuAt41xvzas+kJYGLk8UTg8YwulG7q5WGHWXF2Yu+c/Ze/bMV9y5bgmaqcaLtzptJB68cv9qmEcbziFW8ErTE2PBVUdTKbMfvt22PHDDi8/5/PP8/sGokICuMUi9jnLT0tm3z0EZx3Hhx9NNx4IxBcUsFLZaW9NyhlxbHABOB4EVkS+TsZmAGcICIfAidEnqdHU1NsyCaVQVWVlXDwwVGxd6Mu+/SB7t2TD+Nk4uxdGMfv7MPCOLNmwQMPRF+X9zWGsW2bPa8TQi/ZdPaffRYdR+Cl0M6+R4/o4yyQyQhal572hoh0AxaLyHPAudj0tBkicg02Pe3qzJuaIbt2wZgx9ku1YMHeOQmdY582zXbCupvpli02Tj99urr6csMY8woQNrJuZFYu4ne0/nIJLmMlSOzBhnL++lf72I2a7dnT/gx1HbTp1sbJ1NkHidPMmdCvH5x1VvIx+02RKFk2nP2nn9pj+vVrve3zz63g+/srCin2u3dnXezTdvZ5S0/zkfaAp0svhSVL4N57qX15QMw5IFrYrK7O/mmRMyWn+EXO7+zdzSCe2K9bZ6u01tVZR9+hgxX8TPLsw1Iv/aQas9+0KXpTStbZb9xol9lw9ldcAaNHB29znc5uGdS2XIp9Y2P0l4U3jFNVZd/7InD2e4mXniYigelp6eAGPLn33Q14ggSifN99NrF+6lRqt45K7xyKkk38zt4fsw+bktDh7aStq7OOHuxy06bsp176CQrj+LNxNm+2fwcdZN25i5cm6+yd2O/Z0zqklaqzX7fOFhsLwsXjd+yAffYJblsuxd77y8UbxunSxf4V2tk70k1PSzoX2UPQzFP+AU9+5//k/7wD3/8+Gw8+jgNrfx44F60OmlLyjhMSJ7x+Z59I7L/8Zbv84APrmHv1ss+9YZxE5RKSqXoJ6XfQ/vzncMIJ0T6Fujrb6Zqqs4fWoZxUnf2OHcFxeWg9YCyobbkUe/fLpbq6eMU+k/S0pHORPYQNbHLr/aUO6lZ9xoFTx7DdVPFvqx5g+erwHzI6aErJK05InCP3x+zd9qByCQA1NXb/9eutiDqxTyWMk+vUy08+se1zX66GBuuiU3X20DqUs3NnNLyUjBhu3x6Ny3vx1sT33wxc29q1y4/Y9+tn2+JuiJ072/9/ocsl5C09zUNYmqTrVI11/obf8wOGmvc4ddcDLN/VN61zK0pOcGEc9+H1O3snQGHOvl07Wx5h3brYME7PnlbYXDgmlaqX6YRxXMjG6+zdOpfOuGRJ9Li6uuw5+1RSE7dvtyLqT6H0irjf2bvz9uyZH7Hff397Q3SCX0TOPvfpaT6mT4/OfOZlxw7r6r3u/ALuYDy1XMf1vMDxcc+rc8gqeceJnFfs3Qhab6gjTOzBOsH162PDON2726Ux2auNk4qz99azd2LvUkTBttWJV8eOmTl7d4NLRoidkPvdu7dTNszZ9+yZ2zx7N3rWZQq558Ui9saYV4wxYoz5ijFmWOTvKWNMvTFmpDFmSGS5JSstxXagevtPHA0N1tW7780w3uS3XMLTnMR04gfjdQ5ZpSD4wzhO7MEKbjJiv//+8PHHVoic2PfwDGtJdQRtUD17737uXCJ7U5dbddB6wzjOjb/xRvRcXmffo0disT/ggNhzOXbtsjc2kcRiaExUyP2C7hXxeGKfLWf/2mtw/fXwxz9G13nDOBDNWnJhnGLKxskXtbXR98HPqlXWVOzDNhYwhjp6MYF5mJD7WWWlirxSQFwYxxuzd6La1JS8s3/00djzOGcP6XfQhjn77t2tMHXoEL0xNTVFwz8uZu8P4/zrX9Fz1dVFxat798Rif/DBthxEkLOvrExODHftiobH4jn7sA7anj2jE7pnwoYN8LWv2cdf+hL8v/9nH3vDOFB8zj7fuM7XMCoqoLHRcBfnM4iVnMlD1FHTah/vXLQq9ErBCAvjgBWmZMXeEebsU029DMrGce0K+hXinYzbPy2hc+N79kRfR3199LV17x6/XMLGjTB0aOy5HPGyVbb4ggleEU/V2VdU2HBCNpy9S/088EBYvjzaWVxfD127Rm/U5SL2YYOmgtIuvTQ3wyX8ltN5hGuYwd85Nma7iL1Z6GAppSgIEvtUnb1zgtA6Zg/pDaqKF8ZJRuxdGGfPnlghHzLEfgmds+/UKX6myWef2S/8AQfY/fxiH+bsX3/dZip5Q0dBYr95MzzxRGJn37mzFeJUxH7NmuBa6S4sccwx9r1xNY3q6mybXae326+UxT7eLFGJUiOP4R/czBU8xmhu4cetthsDc+fqjFNKkeDPxnGpl5Ces3dCHObss9FB624o/jCOc/IujGNMa3fdt69tm4vZd+kSrcsfhOuc7dPHHucP44Q5++ees6/htdei67wi7h7Pnm1H1HoHWgU5+86d7U0lWbHftMkOInv44dbb3HtyzDF2+dFHdrl5sxV7N+jM7ZflmH1RiX3YoKkpU1qbDS89qWc+Z7CGA5jEHMLKmujgKaVoCOqgzbaz92bjJBPGcZ2tqTp7J/YujAPRujauI7emxrbRZeN07py82HfvHuzsg8T+73+3y2XLouu8Iu4eO5FfvtwuRbIj9suX2/fDCbkX59iPPtouE4m9uyGWotiHuff6+vBpJoUW5jGBPmzkDOazlfhFNnXwlFIUBOXZZ+Ls3Xm6dm0t4v7HYc7eiX2Ys08UxnHOHqyAge1gBejd24p9qs5+v/2iHcNedu1qHcYxBl591T72in1YGAdsTBes2IaFcSor7f8rmbluXWhmU8BYUifihx9u36uPP462pdzCOOkMbLqGGZzMn7mMW1nM8L3rg1KD072GomSdbMTsu3Wzf/vuG3XUItFQTqJsHO9NQSQq2mHOPlEYx8XsISqmrqyDE3uvs+/UKVzsP/nELvfbz74er7M3JtjZf/CBFdSuXROLvRPjFSvssm/f1s7etdM5bq+7X78+OiWjF5e1Eyb2VVW2zYMGWWdvTGtn78pLdOkCX/0qjBrV+lxpUFRin6i2vJ/jK17kBn7KA4zldn4Ys625ufW5dPCUUjQ4kevf3wpK796xApqM2IMN5TgRdrhQTi6dvbu5BHXQQlTsnNjX1Njjk3X2n3xib0C9erV29o2NNi7vd/YuhDN2rD3eOel4zt6Fcfr0CXb2XbrYmwdExb6+3or15MmtBd+JfVC9r/r66Ht40EHRMRK7d8eK/Qcf2GW/fnDOOdF5ADKkqMR+3DjbbzJwYDRF0r03fob3/4Qnq8byEQcxmdn44/QuvdJ7Lk23VIqGPXusaPbsab/0//VfUVFtbrai5BXVMA480N4wvAQ5+2TKJYTF7N1NKJkOWlfn6v337fJrX7PnGzo0GsZJJmb/ySfR+j+ug/b556NZOhB19u75q6/afU+NVFV37t4JfHV1a2e/fr0V8333jW7buhVeeik2jAPR66xYYW84d94Jt9wS2+5Ezt79kjvoIOvs3U3BG8b58EN7g/P2v2SBohJ7sGLsasuvXGnnPPA79Kouzfxxn7Pp0rCNj371MC2V3WK2OwfvP5cKvVI0eEv27r9/bDZOUxOsXWudnQQnG+xl9my4997YdV5nn8oIWufQw5y9u4n4Y/Yu5NK7N3zhC/axK5Fw1FG2M/Qb37Biv2ePdbjJxOz32y/6erZutRU0f/GLqJP3O/v334dDDoFDD7XPndhv327bW1NjBb25OXZ0ZlWVDYc5sZ85E44/3gpxkNg7QR88GG66KdbdxxN7r7M/8EB7PddGr7NvbrbnzjJFJ/Z+gtz+30/8GfstewFuu41R1xymDl5pewRNOeh19mvWREsFxKNfv9YdUU7sRay4tGtnnWu6zt61q7LS/vnF3oVCBg+OitRbb9nrV1XZm4BrK9gwhXP2DQ3BOemffBIVe+eGO3SAhQtjnX1lZVTsV6ywN5sBA+z6d96x67dvtwOjunWzj10JaEfXrna7C+N88IH9H7z/fjTP3p0HooL+ve9ZUX/77ei5vGEc/+vyOvsvftEuX37ZLmtqYj8P7qaZRYpC7LdsiT/7VIxD//3THPbEdJg0yf75t69UoVfaAHv2REXX4RXQZMU+CG8Y58QT4b33bKgnXget19mHib0TV38YZ8UKG6vff38rmq6WjPcGAzA8kkDhOleduAXNbPXJJzaODvYLfffdcM01drCUS5v0OvuGBvtraPBge80vfQnefdfu5xX7HTtax9Ods3clkN3Ny00L6W6mq1bZ5bp19j05+2z7fOFCuzTGbuvY0d4s/OmiXmfvfn288IJd1tTYdrv3pFSd/apVwQOpWrFmDYwfb9+o3/0u7+1UlKwRz9k3NtrPerqpY/4wzpAh0eeQfgetE3u/s1+xwv6kdudzrnTffWPPM2RItG3O2UPrUI4xsc6+d29r7P7zP60Y//nP0fY4sV+92h7nRPLLX44N43TrZgV/x45oiMU59qoqu82VQHYpka6d7vW49evX27YNGmRf0/PPR6/z+efRTmlvKKelJdbZ9+9vr7l4sX3u+jpcKKdUxd7/a8c7+MmVT+gojSwaMpbGz/fYCcNTSdtRlGIjSOxduOONN6zgZ8PZe0kk9sk6e6/YNzZaJ+wNOzih8ot9u3bRAUVeZ+8X+23b7C8fJ/aOY46xx9x8s73+kCH2PN5BTK4dhxxiXfbWrdHpBv3O/pBD7LJrV7sNrJB7nb+L2fftGxX7deuiIamRI+HFF+374EI4bspIr9jv2GGFzom9iDWtzc32/1FVZdeXutgHsXp1bPmEXzKV4Xv+zvktd1K7aGihm6comREUxjnsMLt88km7TFfsvc7eS7IjaMOcvYvZB4VxvOLkHgdlk7hSAfGcvXdAlZdOnWxHb0MD3HWXjXu7DBYXn/c6e7ChHH8Yx4mwE3sXxgFYujT2mq6NBx4YLPYnnGDDP88/HxX7YcPs0nvTcB3C3vRCF8qpqYl2pLvXU05iP2BAtHzCKTzOFdzC77iIeQ1naskDpe0T5Oy7dbOi4sICuXD23lLKRx1lY/oHHBC7b1jqZZcu8N3vwsknR9fV19vwhFecwsI4ECv27qbjF3vvgCo/N95of9mfc459/qUv2eWDD9qbkCsh4YR82bLgmL1I9FgXxoFoWMWN/PWKvfv14BX7UaPsNW+6KTp61om919m7nH/n7CFW7B3O2Q8a1Pq1Z0hRiL3/s+VSJ1evhsEsZy4TeZ3h/Bib06olD5Q2j7fsr5fDD49mm2Qas/enbfbtG3sDGToUnnnGfuG8X8IwZ9+5M1x7LVx+eVTsP/zQLpMJ40A0jFNVFe7sndi7DlovRxwBp58efT5ypL0JLVpkBdK1ddAgu/6dd6Ix+27doh25PXva9wNsGOeII+z7dc89dt2JJ0ZfM1ixd/P9btsWFftOneCyy2xH6//+r113+OF2+eyz8PWv218XiZy9o7LS3jwSDaZLg6IQ+4EDg1Mnhxywm/mcgUE4g/k0YJ2AljxQ2jy7d7cO40BUKLp0iXWBqRAWxrn00tZhCkc8Z19RYdvqXe8X+yBnHxTGqamBxx6zaYtO0F54IRqGgfjO3k9lpQ2l+NvgMnKWLYuN2YMNx/TuHRXZqior3v/xH9aNd+9uf/VANKxy4IF2+cordumtSzR5sv1fvfee/eXhMpIeewz+8Q8rZu41JXL2/fpFPwNZpihmqurZ096Y/fxxyOV8cfViTuFxVmL/kVryQCkkIvItYCZQAdxpjElvjuWgMA5Ev+gHHJB4QFUYzhE7oXK4OWKDiCf2PXu2dtnxxH7AAHvtMLEePdou3Sjbyy+3vwJcDfJ337Xt7BG/qOFeTjnF1qb3x7kPOcSGxD7/3AqwC9UsX26dv+sQd1k548bBX/5ib1ZO3L3OHuzIWogV+332sTfRysqomNfU2NDNN75hbxDXXmvXe519TQ185SvRsA/YFNOgmjtZoCicfSAPPMAXF97OslFX8tbAU3TAlFJwRKQCuA34NnAIcJaIHJLWycLCOF/5il2mG68HK2TPPhsV1WSIF8a5+uqoyPn32brVjoz1CnPHjvDPf8Ill8S/potPH3mkvTmceio8/bQVezfwJhlGjbLXdJ2yjiOOiDrq/fePOvs1a1o7e4DTTrO/YA480MbsKyujon7QQXb54ot26RV7sKmUXtfeu7e9YT30EPz85zZ0BK1vYEuXwhVXRJ+7FNEcUBTOvhXvvQcXXADHHsshf5jOygTlQRQlTxwNfGSMWQ4gIg8Co4FlcY8KIiyMM2iQFQ3nJNPFhTaSJZ6zd/FuLyJWDKuqYN681r9CXIgiHl/9qi1NMH68FeA5c+yNpaUlmhmUDPvtZzXDL8AXXmgFf5997NIJdU0NXHyx3f/CC+Hb37br993Xzuk7YIAV5dWro+Lcs6cN77gZsPzX8jN5sq13tP/+8NOfwr/9m+3gbV84yS0+sd+5E8aMsXd618OuKMVBP2CN5/la4Bj/TiIyGZgMMCCsg+nb3479+R492IYSkolXZ5OxY+3I1K5d4ZvfTO6Yl16yP7f9VTeTpUOHqPvv2RN+85v0zgPBqYqdO9s4vOO442D+S3+iggAABeJJREFUfNv56jqPb7st9piTT44+9oZcROCOO2xlzUGDor8Gwhg/Pvb5yJH2r4CIyVF8KBWGDx9uFrmg/aRJMHcuC696mvMfPJHVq+2N1hU2U5R0EJHFxpjhifeMe44xwEnGmO9Fnk8AjjbGXBx2TMxnW1GyTCqf6+Jy9nPmwD338NZ3/5tT/vfEvRloroQCqOArBWUt4A2m9wfWF6gtipISxdNB+9ZbNn52/PGcuvi/A+ei1cFUSoF5HRgiIoNFpCMwFniiwG1SlKQoDmff0mLj9D16wP33s7Jv8JyCOphKKSTGmCYR+RHwDDb18m5jzDsJDlOUoqAoxH7LmytpZht/ufYFTujThwEDotVEvehgKqXQGGOeAp4qdDsUJVWKIozTk0+ZxnRO/fUIamuD56LVwVSKoijpkzOxF5Fvicj7IvKRiFwTb9+N9OFGroqJy3sH/1VX62AqRVGUTMiJ2Kc60nAt/TGRprjMG+8UkW7WMUVRFCU9cuXs9440NMY0AG6kYUIqKtBMHEVRlCyTqw7ahCMNvaMMoRNgxwU0NwefcNUqEHHFptscvYC6QjciR7SV1zawEBddvHhxnYgEpBsAxfXeaVuCKfa2JP25zpXYB5Xrixmqa4yZDcwGEJFFxizKaHRjMWNfX2ajN4uVUn5t2cAYUxO2rZjeO21LMKXUllyFcXSkoaIoShGRK7HXkYaKoihFRE7COGmMNJydi3YUEaX8+kr5teWaYnrvtC3BlExbiqLqpaIoipJbimIEraIoipJbVOwVRVHKgIKLfSplFYodETlARF4QkXdF5B0RmRJZ31NEnhORDyPLJGdSLj5EpEJE3hSRJyPPS+a15YtCfubjfEZ/JiLrRGRJ5O/kROfKUntWisi/ItdcFFmX98+UiAz1vPYlIrJdRC7N1/siIneLyCYReduzLvR9EJGpkc/P+yJyUlLXKGTMPlJW4QPgBGy65uvAWcaY1Of0LAJEpC/Q1xjzhoh0AxYDpwLnAluMMTMiX+4expirC9jUtBGRy7Ej4PYxxnxHRG6kRF5bPij0Zz7OZ/QM4DNjzM35aIenPSuB4caYOs+6gn6mIv+jddiBoJPIw/siIiOAz4B7jTGHRtYFvg+R0jMPYCsV7A88D3zRGBMyJNVSaGefdlmFYsQYs8EY80bk8Q7gXexo4tHA3Mhuc7FfrjaHiPQHRgF3elaXxGvLIwX9zMf5jBYThf5MjQQ+NsaEjXzOOsaYl4AtvtVh78No4EFjzB5jzArgI+znKi6FFvugsgrF9sFLCxEZBBwBvAb0McZsAPtlA3oXrmUZ8RvgKqDFs65UXlu+KJrPvO8zCvAjEXkrElLIVzjOAM+KyOJICRUo/GdqLNY5OwrxvkD4+5DWZ6jQYp+wrEJbRESqgEeAS40x2wvdnmwgIt8BNhlj2mp9omKhKD7zAZ/R24EDgWHABuCWPDXlWGPMV7EVci+KhDMKRmQQ6CnAgsiqQr0v8UjrM1RosS+5sgoi0gH7Jao1xjwaWb0xEit1MdNNhWpfBhwLnBKJsT4IHC8i91Eary2fFPwzH/QZNcZsNMY0G2NagDtIIiyQDYwx6yPLTcAfItct5Gfq28AbxpiNkXYV5H2JEPY+pPUZKrTYl1RZBRER4C7gXWPMrz2bngAmRh5PBB7Pd9syxRgz1RjT3xgzCPt/+osxZjwl8NryTEE/82GfUScqEb4LvO0/Ngdt6RrpJEZEugInRq5byM/UWXhCOIV4XzyEvQ9PAGNFpJOIDAaGAP9MeDZjTEH/gJOx2QkfA9MK3Z4MX8s3sD+n3gKWRP5OBqqBhcCHkWXPQrc1w9f5TeDJyOOSem15ev8K9pmP8xmdB/wrsv4JbMZOrtvyBWBp5O8d914U6jMFVAL1wL6edXl5X7A3mA1AI9a5nx/vfQCmRT4/7wPfTuYaWi5BURSlDCh0GEdRFEXJAyr2iqIoZYCKvaIoShmgYq8oilIGqNgriqKUASr2iqIoZYCKvaIoShnw/wGyaVgvNLMzKgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from IPython import display\n",
    "import math\n",
    "\n",
    "def grad(_X, _W, _Y):\n",
    "    return (calc(_X, _W) - _Y) * _W[0]\n",
    "\n",
    "def cost(_X, _W, _Y):\n",
    "    return 0.5 * (calc(_X, _W) - _Y) ** 2\n",
    "\n",
    "def update_W(_W, _cost, rate=np.array([0.003, 0.002])):\n",
    "    _W -= rate * _cost\n",
    "    return _W\n",
    "\n",
    "guess_W = np.array([1.0, 1.0])\n",
    "loss = []\n",
    "for x, y in zip(X, off_Y):\n",
    "    _cost = grad(x, guess_W, y)\n",
    "    guess_W = update_W(guess_W, _cost)\n",
    "    loss.append(cost(x, guess_W, y))\n",
    "    plt.subplot(121)\n",
    "    draw_line(guess_W)\n",
    "    draw_scatter()\n",
    "    plt.subplot(122)\n",
    "    plt.plot(np.arange(len(loss)), loss, color='red', label='loss')\n",
    "    plt.legend()\n",
    "    plt.show()\n",
    "    display.clear_output(wait=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2.08900161, 1.72600107])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "guess_W"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 批量梯度\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU1f3/8dcHVCSuEFERSLAWZZOqpG5oq6i4fLViq63+UKm2gtS1KqJirVap1KXWaq3ggihRtIJKXYvUonVBo4iyClVAXFgEVwSBfH5/nAlMkrkzk5lJMpm8n4/HPCZzc++dc9uHnxzO+ZzPMXdHREQKS4vGboCIiOSegruISAFScBcRKUAK7iIiBUjBXUSkACm4i4gUoJTB3czuNbNlZjYz7tiNZjbXzN4xs8fMbPu4311uZgvMbJ6ZHVlfDRcRkWjp9NzvA46qcWwy0NPdewHvAZcDmFl34GSgR+yaO8ysZc5aKyIiaUkZ3N39RWBljWP/cvf1sY+vAR1jPx8PjHf3te7+AbAA2DeH7RURkTRsloN7nAk8HPu5AyHYV1kSO1aLmQ0CBgFstdVWvbt27ZqDpoiINB9vvvnmCndvl+h3WQV3MxsOrAfKqw4lOC1hfQN3Hw2MBigrK/OKiopsmiIi0uyY2aKo32Uc3M1sIHAscJhvKlCzBOgUd1pH4ONMv0NERDKTUSqkmR0FDAN+4u6r4341CTjZzFqZ2a5AF+D17JspIiJ1kbLnbmYPAYcAO5jZEuD3hOyYVsBkMwN4zd3PdvdZZvYIMJswXHOOu2+or8aLiDRbKSr6pgzu7n5KgsP3JDl/BDAiZcNERCQzixbBuecmPUUrVEVEmop16+Cmm6B7d3jhhaSnKriLiDQF06bBD38IQ4fCYYfB7NlJT1dwFxHJZ198AeecAwccACtWwIQJ8MQTUFKS9DIFdxGRfOQOjzwCXbvCnXfC+efDnDnw05+CJVpSVF0uVqiKiEguffBB6K0/8wzssw/8859QVlanW6jnLiKSL9atgz/9CXr0gJdegr/8JYy11zGwg3ruIiL54dVXYfBgePddOP54uO026NQp9XUR1HMXEWlMn38OQ4ZAnz6wahU8/nh4ZRHYQcFdRKRxuMPDD4cJ09Gj4cILQ3rj8cfn5PYalhERaWjvvx8mTJ99NoynP/10mDjNIfXcRURSKC+Hzp2hRYvwXl6e6ooI69bByJFhwvTll+Gvf4XXXst5YAf13EVEkiovh0GDYHWs/u2iReEzwIABdbjRK6+ECdOZM0Ou+q23QseOqa/LkHruIiJJDB++KbBXWb06HE/LqlUhqPfpA19+CZMmhVWm9RjYQcFdRCSpxYvrdnwjd3jwwTBhes89cPHFMGsWHHdcztuYiIK7iEgSUSVckpZ2WbAAjjwyjNt07gwVFaGa49Zb10cTE1JwFxFJYsQIKCqqfqyoKByvOdH60Njvwi969gwTpbfdFsba99qrwdutCVURkSSqJk2HDw9DMSUlIX5D9YnWToteYq8zBoPPgRNPDBOmu+zSOI1GwV1EJKUBA2pnxnTuHAJ7G1byJ4ZxFnez0Es5o92TjPnH/zVKO+NpWEZEJAOLFzkDGMdcunIGY/gTl9KDWYxd0fiBHdRzFxGpu/nzeXHLIRy0ZgqvsR+H8zzv0guA0uR7aDQY9dxFRGqIXJG6di1cey3suSf7tqjg/M3/zoG8sjGwV0205gMFdxGROFUrUhctCqnqVStSJ185FX7wA7jqKujfny0WzGG/MWdTUtoCMygtDfW/6rRqtR5pWEZEJE7NFanFrODG1UM5YsR9sOuu/HvoM5z5yFEs7rApcyZfAno8BXcRkTibVp46p3M/N3Mx2/EF13M5u155Jb86ryj7OjMNQMMyIlKQMq3kWFICuzOPf9OXsfySeezB3kxnVOkfuewPRdnVmWlACu4iUnCixs1TBvg1a5i0z9W8Qy/2ZjqDGMXBvMQHRT0ZMSKLOjONQMFdRApORpUcX3gBfvADej12DR8feCKHd5zH3TaIktIWGydKM6oz00hSBnczu9fMlpnZzLhjbc1sspnNj723ifvd5Wa2wMzmmdmR9dVwEZEodephr1gBv/wl9O0LGzbAc8+x68vlVHy4E5WVsHDhpvH0ZHVm8k06Pff7gKNqHLsMmOLuXYApsc+YWXfgZKBH7Jo7zKxlzlorIpKGtHrY7jBmTCjJ++CDcMUV8O670K9f5H0HDAjpjqWl5GX6Y7yUwd3dXwRW1jh8PDA29vNYoH/c8fHuvtbdPwAWAPvmqK0iImlJ2cOeOxcOPRTOPBO6dYPp0ynvPoLO3VqnnIAdMCD05mv26vNNpmPuO7n7JwCx9x1jxzsAH8adtyR2TESkwUT2sH+2JixC6tUL3nkH7roLpk6l/O0emU3A5rFcT6hagmOe8ESzQWZWYWYVy5cvz3EzRKS5q9XD3nlKCOrXXsvELX7BTqvm0vm6X1P+UIvst9LLQ5kG96Vm1h4g9r4sdnwJ0CnuvI7Ax4lu4O6j3b3M3cvatWuXYTNERFJYtgxOOw0OP5wvv3SObTWZn33zAMvYcWMPfdGixJfmY4pjujIN7pOAgbGfBwJPxB0/2cxamdmuQBfg9eyaKCKS3qKk+HN2La3ktbPuCROmDz8Mv/sdP2z1Lk+tPbzaNatXQ8uItI98THFMm7snfQEPAZ8A6wg9818BxYQsmfmx97Zx5w8H/gfMA45OdX93p3fv3i4iEmXcOPeiIvcwIh5eRUXheKJzujHLp3KwO/jSPQ52nz3b3d3Nqt+j5v2S3T8fARUeFbujftGQLwV3EUmmtDRxQC4trX7Olqz2axnua9ncV9DWz+Ae71yywceNi75H1X2qzjHb9DnfJQvuFn7fuMrKyryioqKxmyEieapFixCGazILE6YA/WwydzCE7/M/7uc0LuZmVhDm84qKak+YVikqyt9c9VTM7E13L0v0O5UfEJG8l3RR0tKlMGAA/6IflbSgL1MYyP0bA3vLltGBPZ8XIWVLwV1E8l6iRUlbta7k4cNGhwnTRx/lnRN+z/6t3+EF+m48p6goVBRIxCy/FyFlS8FdROpNpmV3a6q5KOmI9jP5oOPB7Hfv4LA70owZ9Jp4NbfdtWWthUulpYnvWVKSu/blpajB+IZ8aUJVpPCkk+FSZ998437ZZe6bbeZeXOw+Zox7ZWVG7RgypGlmyMQjyYSqeu4iklNVveFTT83xqs9nn4WePWHkyLAoae7cUM3REi2M3ySqFMHTTxfeqtR4ypYRkZyp2iQjagITqme4pOWTT+C3vw0LkfbYA+68Ew45JNumppWBk++ULSMiDSJRjZaa0l71WVkZAnm3bvD443DNNTBjRk4Ce7J2NOlVqXEU3EUkZ1LVYkl7Y4t33oE+fWDIEOjdO3y+6ipo1Son7YSmtfFGJhTcRSRnkvV608op/+YbGDYM9tkHFiyAsWPh+edh991z3tamtPFGJhTcRSRtqVIHo3rD48ZVzylPeJ+nn4YePeCGG8JE6dy5cPrpKSdMs0lnbCobb2QkKo2mIV9KhRTJf+mmNqaq0VLzPu35yB9teVL40K2b+9SpOW9ToUKFw0QkW+kU3qrLfVqw3n/D7f452/q3tPIbtr/Ofe3aauem+kORTkGxQpYsuGtYRkTSkmyytC7b0i1eDL2YwSscyN84l2nsR09mMuyL4bDFFhvPq0qrTLb1XVSbmvImG7mi4C4iaUmVIphqAVB5OXQr+YY/+VDepDedWcj/o5wjeY7/8f1a909n67tCT2fMhoK7iKQl0WRpTVE95vJymPCrp3j2w+4M5Sbu5Uy6MYeH+H+AJUxBTNYrr5pEXbSo9nxrIaUzZkPBXUTSEp86GCVhj/mjj2gz6EQmrj2Wr9iGPvyXwYxmFW2B6BTEqN5327bV9z113xTgCy2dMRsK7iKStqrUwXHj0lgAtGED3HYbdOvGoauf4gpGsA9v8Qp9Np6SrOxuVFol1B6ucQ+BveDSGbOg4C4idZZyAdD06XDAAXD++XDAARy5y0yu5wrWsUW1+yQbG4/6jpUrE5+vSdTqFNxFJCMJFwB9/TVcdBGUlYVo++CD8OyzDL5ht4yW+if6Dk2ipkfBXURyY9Ik6N4dbrkFzjoL5syBU04Bs5wu9S/0mjC5slljN0BEmrglS8Lwy2OPhXrr48fDgQfWOm3AgNyMh1fdY/jw8I+DkpIQ2DXWXp2Cu4hkZsMGuP12uPLK8PPIkWFIZvPN6/2rc/WHopBpWEakGUinuFadCnC9+Sbstx9ceCEcdBCPj5hF578Po0WrzQtvL9KmKqouQUO+VFtGpP6kU1wr7QJcX37pfsEF7i1auO+8s/v48T7ugcqsi3fF15ApLg6vqHoysgkqHCbSfKVTXCvlOZWV7hMnunfoEKLub37jvmpV2vdPJtEfluZa5bGukgV37aEqUuDS2Ss06TkLF8N554VsmF69YNQo2H//Ot0/maoyAslULVCS6rSHqkgzlk5eeKJzWrKea7b/c0hvfP75sIlGRUW1wJ7u/ZNJZ/GRFijVXVbB3cx+a2azzGymmT1kZluaWVszm2xm82PvbXLVWBGpu3TywmueU8YbvGk/5HerLg4bUs+eDUOHJsyEyTbvPJ0/AlqgVHcZB3cz6wCcD5S5e0+gJXAycBkwxd27AFNin0WkkaSzgKjqnB6dvuQ2zmMa+9Flu6Xw6KPwz38mrRaW7QKlVNUmtUApQ1GD8aleQAfgQ6AtIV/+SaAfMA9oHzunPTAv1b00oSqSmWyzTDZeT6UP3uFR/6bNLuEG557r/vnn9dz6BO1QtkydUF/ZMsAFwNfAcqA8duzzGuesirh2EFABVJSUlDTE/w4iBSXbLJOq60tY6JM41h38bdvLn7lmWsM9hGQlWXDPOFsmNpY+AfgF8DnwD+BR4HZ33z7uvFXunnTcXdkyInWXbZbJbqXr6b/4Vv7AVQD8jmv5K+fTsXQzZaY0EcmyZbIpP3A48IG7L499yUTgQGCpmbV390/MrD2wLIvvEJEIWWWZTJvGhMWD2YsZTOI4zuM2FlOa9n0l/2WTLbMY2N/MiszMgMOAOcAkYGDsnIHAE9k1UUQSySjL5Isv4Jxz4IAD2Lnlcn7KBI7niY2BPd37Sv7LOLi7+zTCMMxbwLuxe40GRgJHmNl84IjYZxHJsTplmbjDP/4B3brBnXfCeecx9e9zeK7op4AlvkaatqjB+IZ8KVtGJDNpZZl88IH7MceEWdZ99nF/442E18dfE3Vc8gsqPyDSDK1bFzbOuPpqaNkSrrsuDMlslnyqrbw8bEAdv09pUZE2ns5HKj8g0ty8+ir07g3DhsGRR4YVphdckDKwQ9gEo+YG1KtXh+PSdCi4ixSSzz+HIUOgTx9YtSrsjvTYY9CpU9q3iMqWURZN06LgLlII3OHhh6Fr1zB+csEFobfev3+db6UNqAuDgrtIU/f++3D00XDyydCxI7zxRhhr32abjG6XKAvHLCyY0i5LTYeCu0hTtW5d2Le0Rw94+WW49VaYNg322Ser28YXAoMQ2KvyLhYtCpOtCvD5T8FdpIkpL4ef7fwyM7fYGy6/nMV7HgNz5sD554esmLjz0t4TtYYBA0LZgtLS2htxaHK1aVBwF2lC/jF6FWsHDmLC0oPYhq84jkl0mzWB8qkdq51Xlc64aFEIzpn2uDW52nQpuIs0Be7w4IMcMqQrp2+4l5u4mB7M4kmOS9iTTpbOWJcevSZXmy4Fd5F8t2AB9OsHAwbwQWUpP+QNhnIT37D1xlNq9qSjetZVPfh0e/TZ7rIkjUfBXSQHshnfjrz2u+9CFO3ZM0yU3n47J5e8ytvsXeseNXvSUT3rli3rtkAp212WpBFF1SVoyJdqy0hTlmjTjFQbZaS69rkrX3Tv1i0cOPFE948+qtN3RZ0XtbGHWT38DyP1jvraiSlXLwV3acpKSxMHzNLSul/bhs/8Ln616QZPPlnrmnSLeiU6L5u2Sv5RcBepR2bp9YYTBdtN11b6qdzvS2nn62jpNzDUH7r765xXZszmXxmSfxTcRepROr3hqKBaXOzehXk+mcPcwV9lP9+TGV5cXH9BWOV8C0ey4K4JVZEspZNRkig1cf3qtVy8+lreoRdlVHA2f+dAXuF/Rb2A+qvMWLVAqbIyvGtytDApuItkKZ2MkpqpiT9iKm+zF5d/exWf7t+fIzrMYbSdTUlpC0aPhpUrE3+XFg9JuhTcRZKIT1PcYYfwSpTumKo3XJWa2JbPuIczmcohtGItA3d8hs6vjueNJe2rXavFQ5ItBXeRCDWX8H/2WXh5Bsv5R1zn/HqL+5lLV07jAa7nMvZtPZN+fz4q8flaPCRZUnAXiZBonDxeOmPg5eVwaIf3aH/aYdz13UDe32x39mE6o0qv59a7iiLHu2sO9RQXQ+vWcNppKrsr6VFwF4mQzvh2snMeum8tH5xxDc9+vCd7M51BjOKwzV/isnE905rIrBrqeeAB+PbbzP/VIM2TgrtIDVXj7J7G3vGJxsDLy+Hknf/D3mf8gCvXXc0EfkY35nAXg/jm2xZ1znjRnqaSCQV3kTjx4+ypJBoDf/TOFfjAXzJ+6aFszjr68RwDeJCl7LzxnGS9/UR1ZlR2VzKReit0kWYk2Th7cXF4X7ky9NhHjIgbWnGH++6j7zlD2abyC0ZwBddxJWtoXes+URkvVX9Yqr6/avilbdswJJPufURAwV2kmqjesBmsWBFx0dy5cPbZMHUqs+nDYEYxmx4JT02W8RI1/NK6dbgu/nfKnJFUNCwjEqdO+eVr1sBVV0GvXjBjBtx1F6eVvBgZ2FOVy436w7JypcruSt0puEtByrS+etr55VOmhKB+7bVw0kmh9/7rX3PdH1skvH7cuNRL/ZP9YVHJAKmzqKIz6byA7YFHgbnAHOAAoC0wGZgfe2+T6j4qHCa5lG3lw6SFtZYudT/11HDT73/f/V//qtv19dhuaX5IUjjMPJ18rwhmNhZ4yd3vNrMtgCLgCmClu480s8tiwX1YsvuUlZV5RUVFxu0Qide5c+Jsl9LS0OvNSGUljBkDQ4fC11/DpZeGQfLWtSdMs1FeHm67eHGCSVuRGszsTXcvS/i7TIO7mW0LzAC+53E3MbN5wCHu/omZtQf+4+57JLuXgrvkUosWiXPUzUKMrrPZs8OE6UsvwcEHw6hR0K1b1u0UyVay4J7NmPv3gOXAGDObbmZ3m9lWwE7u/glA7H3HiEYNMrMKM6tYvnx5Fs0QqS7dSdGURcG+/RauvBL22gtmzYJ77oH//CcngT2bPVdF0hI1XpPqBZQB64H9Yp9vBa4FPq9x3qpU99KYu+RSOmPXic6Jfx3X6jn/csfvhQ+nn+6+bFmDtk8kHdTTZh1LgCXuPi32+VFgH2BpbDiG2PuyLL5DpM7Sqa8etVhpR5YyjgFMWnsky1e2DFkxY8dCu3Y5a5/KCUhDyDi4u/unwIdmVjWefhgwG5gEDIwdGwg8kVULRTKQKnWwZk65UclZjGYuXTmRR7mGq+ix/h3o2zfyOzIdWlE5AWkI2a5QPQ8oj2XKvA+cQfiD8YiZ/QpYDJyU5XeI5FxJyaaMmh7MZBSD6cMrvMAhnM2dvMcelJZGXx9VKgBSZ7fEf3fN4yK5ktUiJnd/293L3L2Xu/d391Xu/pm7H+buXWLvERuGiaQvqpecTu850TkjRkBx69WM4Aqmszd7MI9fMoa+/Jv32CPl8v5shla0EYc0iKjB+IZ8aUJVkomagBwyJLOJ06Ii9ylDn/Ev2+3qDn4vZ3iXNsu9uDj9hUdmiSdizdJ/pkwWOonEo74WMeWK8twlmahFSS1bwoYNtY/HL1aqee1OfMot/JZTGA977AF33gmHHJKzNmW1UEqkjuorz12kQURNNCYK7DXPr/rZqGQwdzKXrpzAY/yea0KxrwwCO2hoRfKfgrvkvaiJxpYtU59fUgI9eZf/chB3MoQ36U0v3mFs6VXQqlXGbUon3VKkMSm4S96L6iUPGpSi9/zNNzy15zDeYh+6MJ/TuJ/DeZ6PinbPSQ9blRolnym4S96qynI57bRQn6u4uHov+Y47kvSen34aevakx5M3sOjHp3NEx7mU22kUFxutW4d7atm/FLSomdaGfClbRmrKeIn+Rx+5n3RSuKBrV/epU7O/p0ieQtky0tTUORtlw4ZQrfHyy2Ht2pBwfuml1cbVleEihSZZtoz2UJW8VKcl+jNmwODBMG0aHH54GK/p0iW7e4o0cRpzl7yUVtneb74Jm2f07g3vvw8PPAD/+lfCwJ72PUUKhIK75KWUeeRPPQXdu8NNN8GZZ4Y9TE89NcysZnpPkQKi4C55KTKP/JCP4MQT4dhjYeutw+5Io0dD27aZ31MpjFKANKEqTcOGDWEsffhwWLcOrroKLr4YttiisVsm0mg0oSpN2/TpYcVSRQX06xeC/G67NXarRPKahmUkf339NVx0EZSVwYcfwkMPwbPPUv7abtp/VCQF9dwlPz3xBJx3XgjqgwfD9ddDmzZZbZIh0pyo5y755cMP4YQToH9/2G47ePnlUJa3TRtA+4+KpEvBXfLDhg1w660hvfG552DkSHjrLTjwwGqnaSGSSHoU3KXxvfkm7LsvXHghHHQQzJoFw4bB5pvXOlULkUTSo+AuSaWzR2nGvvoqBPR994WPP4bx40M1x113jbwk2UKkem2rSBOjCVWJVG+Tl+7w+ONhwvTjj2HIEPjjH8MYewpV3zt8eBiKKSnZtMJUE60im2gRk0SqlyqKixeHoD5pEvTqFZaI7rdfFq0MVPFRmiPtoSoZyenk5fr1cMstYcL0+efhhhvCoqQcBPZkbdJEqzRXCu4SKWeTl2+8EcbVL7oobEg9e3ao5phgwjTTcXNNtIpUp+AukbKuovjll3D++aF3/umn8I9/wD//GcZKEqga41+0KAzLV42bpxPgVfFRpIaoLZoa8qVt9vLXuHHupaXuZuE9rS3pKivdJ0xw32WXcOG55/rDoz9PeZ/S0upb4FW9Skvrsa0iTRjaZk8azKJFcO658OSTsNdeMGoU5fP3rZbJAqFXXbPcbosWIZzXZAaVlfXfdJGmpl4nVM2spZlNN7MnY5/bmtlkM5sfe2+T7XdI7tRbLvj69XDzzWHC9N//Dj/HxtqjSgacemr1NmjcXCR3cjHmfgEwJ+7zZcAUd+8CTIl9ljyQzZh2Ms9eM43ZRWVwySU87315/I+zw+TpZmEZRbKMlUWL4LTTQu/8669rl2fXuLlIZrIK7mbWEfg/4O64w8cDY2M/jwX6Z/Mdkjs5L7r1xRfM63cu/a4+gG3XreAEJnLEt5MYcEVptT8YqXreVUMxn30Wfi4u1k5JItnKtuf+F+BSIH5EdCd3/wQg9r5jogvNbJCZVZhZxfLly7NsRvNU1yGWnOWCu4fMl27d+P7kv3Mb59Gd2TzOCYDV+oORKJMlyrp1Yfe8ysqw+EiBXSQzGQd3MzsWWObub2ZyvbuPdvcydy9r165dps1otjIZYsnJmPbChWH/0p//HNq3Z3+mcSG38hXbVjst/g9G/N6l6dDCI5HsZdNz7wP8xMwWAuOBvmY2DlhqZu0BYu/Lsm6l1JLJEEtWueDr1oVVpd27w9SpVJx6C7utmEYFCSfqa/3BGDAg/F0YNy51L14TqCI5EJUjWZcXcAjwZOznG4HLYj9fBtyQ6nrludedWeKccLPk12WUC/7qq+577hm+oH9/n3jrYi8qSvz94F5UlPy+VW2oam9drhWRTUiS514fwb2YkCUzP/beNtX1Cu51l+2Cn7SsWuV+9tkhAnfs6P7YY0m/G9yLi8PLrPrPUX9ItPBIJHPJgrsWMTVRNcvxQuKFQRlxh4cfDrXWly+HCy6Aa66BbbYBohcbVbWh5nBRztsnIoCqQhak+EnKTNMGE2bbvP8+HH00nHIKdOwYFiL9+c+wzTYbz48K7C1bRgd20F6nIg1Jm3U0YQMGZN4Lrtnz/3jRd8w748+s5xo223LzsJ/pOeeEiJ3g/JqS9djjKRNGpGGo556n6nvLuPhsmwN5mbfYhz+su5wn1h3D/tvOprz4/I2Bveb5NVX9qyGdVEdlwog0DPXc81C9bW8XZ/FiaMNKRnIZg7iLRZRwHJN4kuPgI3i3xvdF9bjNqu90lKp3r1ICIg1DPfc8lG6hrYy5c27xg8yhG2dyLzdyCT2YFQJ73PfFj4+nswCq5jxAcbFKCYg0FmXL5KFk2SgQgqV7CJgjRtQxYC5YEDakfv55Klr8kLMqR/E2e0eeXvUdUI/ZOSKSEWXLNDHpFtqqU1XH774LUXrPPeH11+H223nvvldZVRod2OO/A7LPzhGRhqOeex5KlZlSU2lp9XHvWv7733DDOXPgpJPgL3+BXXap0/el/A4RaXDquTcxOSu0tXIlnHUWHHxwiNxPPQWPPFItsKf7fUphFGlaFNwbWVTKY1aFttzDhV27wpgxMHQozJoFxxwTeY+q74sK8EphFGlaFNwbUTple2v2qs2q36NWeuH8+XDEEWF7o912g7feCtUct9oqrTZlVTlSRPJHVNGZhnw118JhmRT/iiy0tWaN+x/+4N6qlft227nfcYf7hg0ZtUvFvESaBlQ4LD9FpTyahZ2I0jZ1Kpx9NsydC7/4BdxyC7Rvn7N2ikh+0oRqPcu0VEDWOyN99hmceSYccgisWQNPPw3jxyuwi4iCe7Yy2e6uSqLxbbNwj6R/JNxh7NgwYfrAAzBsGON/N4vOQ46mRQvYYYfwqq+6NCLSBESN1zTkqymPuac7bh41jl3nXYnmznU/9NBwwgEHuL/zjo8b51ntjCQiTRMac68/6Yybp7OxRufOocde08bFQ2vXwsiR8Mc/hotHjgw57C1aRF6b8D4iUjCSjbkruGcpZVBO85ykfySmvBAmTN97D04+OUyY7rzzxnNS1aLZeJ+6TNKKSN7ThGo9SpYXXjXRGtWrjl/1mWgStZgVPFL0S+jbF9avh+eeg4ceqhbYo66tyV3j7yLNiYJ7lqK2u4NNE61R4oNy9T8Szi8Zwzz24KdryuGKK2DmTOjXL+F9Ev2BSaROhcZEpEnTsEw9STUOnqhcbtKSNo4AAAvISURBVHk53HvpXK76eDA/5kWW7d6HHSeOgh49Un5feXmov754MbRtG4599lniczX+LlIYNCzTCJIV2kpYLnfNGgbMu4opy3vx4+3fgbvuYsc5L6YV2GFTbZjKSlixIrxqlipIp20iUhi0zV49KSlJPYm60ZQpYQON+fNDlL75Zthpp3prg4qAiRQ+9dzrSVoFuJYtCwW+Dj88zHhOnhyqOeYgsKfdBhEpSAruaapriYGoidYBAwhjJ3ffHVaYPvwwXHklvPNOCPI5lLQNIlLQNKGahnQWIaVt9mwYPDjsjnTwwTBqFHTrltP2ikjzoAnVLA0fXnsLutWrw/G0fftt6KHvtVcI8PfcA//5jwK7iNSLjIO7mXUysxfMbI6ZzTKzC2LH25rZZDObH3tvk7vmZifT6o1R2SVpZ51Mnhw2ph4xIqwwnTs3VHNsob+tIlI/soku64GL3b0bsD9wjpl1By4Dprh7F2BK7HOjy6Z6Y8aleZcuDeM2/fqFQD5lCtx/P7RrV+f2i4jURcbB3d0/cfe3Yj9/BcwBOgDHA2Njp40F+mfbyFxId2glUe++zlknlZVw111hyOXRR+H3vw8Tpn375uhpRESSy8m4gJl1BvYGpgE7ufsnEP4AADvm4juylc7QSlTvHuqQdTJzJvzoR+HCXr1gxgy4+mrYcstcP5KISKSss2XMbGtgKjDC3Sea2efuvn3c71e5e61xdzMbBAwCKCkp6b0oVc3aLOWqemOk1avhuuvgxhthu+3CQqTTT49eJioikqV6y5Yxs82BCUC5u0+MHV5qZu1jv28PLEt0rbuPdvcydy9r1wBj0OkMrWQ8cfrss9CzJ1x/PZx6apgwHThQgV1EGk022TIG3APMcfc/x/1qEjAw9vNA4InMm5c76SzoqfPE6aefwimnwNFHwxZbwAsvwJgxYY87EZFGlE3PvQ9wGtDXzN6OvY4BRgJHmNl84IjY57wQX1xr4cLaY+ZpT5xWVsKdd4YVphMnhjH1GTPCRtVpyDQlU0QkXRkXDnP3/wJR4w6HZXrfxlQV7KtK55aUhMBe7Y/Au++GydLXXgvZL3//O+y+e9rfUXO1a/ykrcoCiEiuaBVNDZG9+2++gWHDYJ99WDNrARcVj6XFv5+nc7/d69TzzslqVxGRFFTyNx3PPAO/+Q0sXMiCH/+KQ1//E0u+Kgbq3vPOerWriEga8rLnnjdj0h9/DD//ORxzTMhTnzqVwxfezZJvi6udFt/zTtX2jFe7iojUhbs3+qt3795eZdw496Ii97CMKLyKisLxBrN+vfvtt7tvu617q1bu117rvmaNu7ubVW9b1cssvbbnxfOJSEEAKjwirjZ6YPcawb20NHHwLC1N/pDjxoVzzMJ7VbCMOh5p+nT3ffcNX3rYYe7vvVft18nal27b69wmEZEEmlRwT9YzjhLVGx4ypA695K+/dr/kEveWLd3btQsnVVam/V3jxmXWdhGRTCUL7nk35p7JmHRUBsro0Wlmpjz5ZNiI+qab4IwzwgrTAQMSrjBNthhK4+kiki/yLrhnsu9nVKbJhg0pzv/oIzjxRDjuONhqK3jppVDNsW3bpG2MSpfUnqUiki/yLrhnsu9nVM+4ZcvExzt32gC33x5K8j71FG+fNIIuX0+nxY8OqpbhktN9U0VEGlLUeE1DvuLH3DNRlzH3A7Z8y1fsWhY+9Ovnj9+8IPvxehGRRkBTGnPPRFSP+Y47Nh3fmq+5a5uL+O/aMopXfwgPPQTPPssFf90tu/F6EZE81CSCe/zwyA47hFfNoZKocfABA2DhrU/wVafu/PqrW2gx6CyYMyfsZWqW+Xi9iEgey/vgXnN3pM8+Cy9PZx/UJUvghBOgf/+wgcbLL4dqjm027R1S1/F6Zb6ISFOQN8E9avIyUZpjvIRDJRs2wK23hgnT556DkSPhrbfgwANrfd+iRbUzHouKwh8NZb6ISJMVNRjfkK9dd+1d54VBkYuEKirce/cOvzj6aPf33681CZFoArbqe7Ja3Soi0oBIMqGa9R6qudCqVZl/911FreOlpeE91faqpaWw8N2v4He/g9tugx13DD33k05KuBApq71SRUTyRL3toZor332X+PjixYkXBsUrKoKxJzwO3bvDX/8KZ58dVpj+/OeRe5iq7K6IFLq8CO5bbJH4eElJ7TTH4uLwMoMDOixmbtfj+fFfTgirSl95Bf72tzB5moTKBIhIocuL4N6hQ/LJy/g0xxUrYMWn66m8+RZe+bw7neY+DzfeCBUVsP/+aX2fygSISKHLi+D+wQfQuvWmHnnSZfsVFbDvvnDRRfDjH8OsWXDJJbD55ml/n8oEiEihy4sJVbMyhwqKipIE2S+/hCuvDMMuO+0Uxtd/9rPIcXURkUKX9xOqVRLmrLvDhAkhZ/3222HIkLDC9MQTFdhFRCLkVXCHGhkrixbBT34SAnm7dvDaayHAxyZMoxY+5c0erCIijWSzxm5ATSUlwPr1IU/9qqvCwZtvhvPPh802NbeqLEHV6tWqUgQvvwxjx9Y+DhpTF5HmI+/G3CdcOo2jHhsMM2aETTRuu23TaqY4UQuRWrZMXPRLC5REpNAkG3PPm557z05f8GjX4exxzR3Qvj1MnBgKftVxIZKqOYqI5Elw7/29VVR82w2e/xTOOw+uvRa23TbpNSUldeu5a4GSiDQn+TGh+v77obf++uthrD1FYIfohUiq5igiUo/B3cyOMrN5ZrbAzC5LenKnTjBtGpQlHDpKKJ3dl7RASUSaq3qZUDWzlsB7wBHAEuAN4BR3n53o/LKyMq+oqF0VUkREojXGIqZ9gQXu/r67fweMB46vp+8SEZEa6mtCtQPwYdznJcB+8SeY2SAgloHOWjObWU9tyVc7ACsauxENTM/cPOiZG07tPPGY+gruifIXq43/uPtoYDSAmVVE/dOiUOmZmwc9c/OQj89cX8MyS4BOcZ87Ah/X03eJiEgN9RXc3wC6mNmuZrYFcDIwqZ6+S0REaqiXYRl3X29m5wLPAS2Be919VpJLRtdHO/Kcnrl50DM3D3n3zHlRW0ZERHIrP1aoiohITim4i4gUoEYP7nUqU9BEmdm9ZrYsPpffzNqa2WQzmx97b9OYbcwlM+tkZi+Y2Rwzm2VmF8SOF/Izb2lmr5vZjNgzXxM7XrDPXMXMWprZdDN7Mva5oJ/ZzBaa2btm9raZVcSO5d0zN2pwj5Up+BtwNNAdOMXMujdmm+rJfcBRNY5dBkxx9y7AlNjnQrEeuNjduwH7A+fE/n8t5GdeC/R19x8AewFHmdn+FPYzV7kAmBP3uTk886HuvldcbnvePXNj99ybRZkCd38RWFnj8PHA2NjPY4H+DdqoeuTun7j7W7GfvyL8h9+Bwn5md/evYx83j72cAn5mADPrCPwfcHfc4YJ+5gh598yNHdwTlSno0EhtaWg7ufsnEIIhsGMjt6demFlnYG9gGgX+zLHhibeBZcBkdy/4Zwb+AlwKVMYdK/RnduBfZvZmrIwK5OEzN/ZmHSnLFEjTZWZbAxOAC939S4vYVatQuPsGYC8z2x54zMx6Nnab6pOZHQssc/c3zeyQxm5PA+rj7h+b2Y7AZDOb29gNSqSxe+7NuUzBUjNrDxB7X9bI7ckpM9ucENjL3X1i7HBBP3MVd/8c+A9hnqWQn7kP8BMzW0gYUu1rZuMo7GfG3T+OvS8DHiMML+fdMzd2cG/OZQomAQNjPw8EnmjEtuSUhS76PcAcd/9z3K8K+ZnbxXrsmFlr4HBgLgX8zO5+ubt3dPfOhP92/+3up1LAz2xmW5nZNlU/A/2AmeThMzf6ClUzO4YwbldVpqDgNsQzs4eAQwhlQZcCvwceBx4BSoDFwEnuXnPStUkys4OAl4B32TQWewVh3L1Qn7kXYSKtJaHT9Ii7/8HMiinQZ44XG5a5xN2PLeRnNrPvEXrrEIa1H3T3Efn4zI0e3EVEJPcae1hGRETqgYK7iEgBUnAXESlACu4iIgVIwV1EpAApuIuIFCAFdxGRAvT/AYDXslu4PnkZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "times = 100\n",
    "batch = 10\n",
    "length = len(X)\n",
    "guess_W = np.array([1.0, 1.0])\n",
    "for _ in range(times):\n",
    "    batch_index = np.random.choice(length, batch)\n",
    "    _cost = 0\n",
    "    for index in batch_index:\n",
    "        _cost += grad(X[index], guess_W, off_Y[index])\n",
    "    guess_W = update_W(guess_W, _cost / batch)\n",
    "    draw_scatter()\n",
    "    draw_line(guess_W)\n",
    "    plt.show()\n",
    "    display.clear_output(wait=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cost(_X, _W, _Y):\n",
    "    return 0.5 * (np.dot(_W, _X) - _Y) ** 2\n",
    "\n",
    "def grad(_X, _W, _Y):\n",
    "    return (np.dot(_W, _X) - _Y) * _W[0]\n",
    "\n",
    "def update_W(_W, _G, learn_rate=np.array([0.001,0.002])):\n",
    "    return _W - (_G * learn_rate)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_X = np.array([X, np.ones(X.shape)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "guess_W = np.array([1.0,1.0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(batch_size, _X, _Y, times):\n",
    "    _,length = _X.shape\n",
    "    guess_W = np.array([1.0, 1.0])\n",
    "    loss = []\n",
    "    for cursor in range(times):\n",
    "        index = np.random.choice(length, batch_size)\n",
    "        x_data = _X[:,index]\n",
    "        y_data = _Y[index]\n",
    "        _G = grad(x_data, guess_W, y_data).mean()\n",
    "        loss.append(cost(_X, guess_W, _Y).mean())\n",
    "        guess_W = update_W(guess_W, _G)\n",
    "        plt.subplot(121)\n",
    "        draw_scatter()\n",
    "        draw_line(guess_W)\n",
    "        plt.subplot(122)\n",
    "        plt.plot(np.arange(len(loss)), loss, color='red', label='loss')\n",
    "        plt.legend()\n",
    "        plt.show()\n",
    "        display.clear_output(wait=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXxU5fX48c9JWAMiEJAtJlGLdRcV+dbaopW6lLYutbIYEVERcfmiVauYWqo21VZccMGKgEQJW12qP7cWrS3qtyqLoFBUqBIMIEsQAdmT8/vjmSGTZG4yme3OTM779ZrXzNy5984zYTh58tzznEdUFWOMMZkry+8GGGOMSSwL9MYYk+Es0BtjTIazQG+MMRnOAr0xxmQ4C/TGGJPhGg30IjJVRDaIyNKQbfeJyCci8pGIvCAiHUNeGysiK0XkUxE5O1ENN8YYE5lIevTTgHPqbJsLHKOqxwGfAWMBROQoYAhwdOCYiSKSHbfWGmOMabJGA72qzgM219n2d1XdF3j6HpAXeHweMEtVd6vqF8BKoF8c22uMMaaJWsThHJcDswOPe+ECf1BFYFs9InIVcBVAu3btTjriiCPi0BRjwlu4cOEmVe2a7Pft0qWLFhYWJvttTTMR6fc6pkAvIsXAPqAsuCnMbmFrLKjqJGASQN++fXXBggWxNMWYBolIuR/vW1hYiH23TaJE+r2OOtCLyHDgZ8AArSmYUwEcHLJbHrA22vcwxhgTu6jSK0XkHOBW4FxV3RHy0kvAEBFpLSKHAL2BD2JvpjHGmGg12qMXkZnA6UAXEakAxuGybFoDc0UE4D1VvVpVl4nIHOA/uCGda1W1KlGNN8YY07hGA72qDg2zeUoD+5cAJbE0yhhjorF3714qKirYtWuX302JqzZt2pCXl0fLli2jOj4eWTfGGJMSKioqOOCAAygsLCQw2pD2VJXKykoqKio45JBDojqHlUAwmW3rVrjySr9bYZJk165d5ObmZkyQBxARcnNzY/orxXr0JnOpwlVXwbPP+t0Sk0SZFOSDYv1M1qM3mWvSJJg9G+6+2++W1Hf//fDii363wjQTFuhNZlqyBMaMgbPOgltv9dxNRFaJyMcislhEFgS2dRaRuSKyInDfKWT/+BTte/hheOGFqA83qat9+/Z+N6EeC/Qm82zfDoMGQefO8MwzkNXo1/xHqtpHVfsGnt8GvKmqvYE3A8/jW7SvY0fYsiWqQ41pKgv0JrOowujRsHIlzJgBBx0UzVnOA0oDj0uB80O2x6donwX6jKeq3HLLLRxzzDEce+yxzJ7tSoKtW7eO/v3706dPH4455hjefvttqqqquOyyy/bv++CDD8a1LXYx1mSWp56C6dPhrrvg9NMjOUKBv4uIAk8EajB1U9V1AKq6TkSCvy0iKtoXWrAvPz8//Lt27AjlvpTfaT5uuAEWL47vOfv0gYceimjX559/nsWLF7NkyRI2bdrEySefTP/+/ZkxYwZnn302xcXFVFVVsWPHDhYvXsyaNWtYutQt+7Elzp0A69GbzLF0KVx3HZxxBtx+e6RHnaqqJwI/Aa4Vkf4N7BtR0T5VnaSqfVW1b9euHoUFO3aEb76JtI0mDb3zzjsMHTqU7OxsunXrxmmnncb8+fM5+eSTeeqpp/jd737Hxx9/zAEHHMChhx7K559/zvXXX8/rr79Ohw4d4toW69GbzPDtt25cvkMHKCuD7MiGzlV1beB+g4i8gBuKWS8iPQK9+R7AhsDu8Svad+CBNnSTaBH2vBOlptZjbf3792fevHm88sorDBs2jFtuuYVLL72UJUuW8Le//Y3HHnuMOXPmMHXq1Li1xXr0JmWVlUFhobuWWljonnu69lr45BO3U/fuEZ1fRNqJyAHBx8BZwFJccb7hgd2GA8E8yPgV7Qv26KurozrcpL7+/fsze/Zsqqqq2LhxI/PmzaNfv36Ul5dz0EEHMXLkSK644goWLVrEpk2bqK6u5sILL+Tuu+9m0aJFcW2L9ehNSiorc3OddgRqo5aXu+cARUV1di4tdbc77oABA5ryNt2AFwKTUVoAM1T1dRGZD8wRkSuA1cBFAHEt2texo7twvG2b692bjHPBBRfw73//m+OPPx4R4U9/+hPdu3entLSU++67j5YtW9K+fXuefvpp1qxZw4gRI6gO/OK/55574toW8frzIpls4RFTV2Fh+GuVBQWwalXIhuXLoW9f6NcP3njDc8hGRBaGpE8mjed3e+pUuOIK9yG9LtiaJlu+fDlHHnmk381IiHCfLdLvtQ3dmJQSHK7xSkhZvTrkyY4dbly+XbsmjcunhGAv3sbpTRLY0I1JGXWHa8Kp1fkdMwaWLYPXX4eePRPevrjq2NHdW6A3SWA9epMyiosbDvI5OVASXOlgxgyYPBnGjnVlDtKNBfqESYXh6HiL9TNZoDcpo9awTB0FBa5GWVER8NlnMGoU/OAHcOedSWtfXAUDveXSx1WbNm2orKzMqGAfrEffpk2bqM9hQzcmZeTnR3ABdudOuOgiaN0aZs6EFmn6FbYefULk5eVRUVHBxo0b/W5KXAVXmIpWmv4vMZmopKT+GH2t4RqAG2+Ejz6CV16BGL74vgvOfLRAH1ctW7aMehWmTGZDNyZlFBW54Znc3JptbduG7DB7NjzxBNxyCwwcmPT2xVXLli5byAK9SQLr0ZuUs3NnzePKStfLb//VSs67cyScckqdLn4aswqWJkmsR29SSrjMm6oduzjs9kFuPH7WLNcbzgRW2MwkifXoTUoJl3kznps5Zs+H8JcXM2sWqRU2M0liPXqTUurG8V/wHNfxGJMPuBHOPdefRiWKDd2YJLFAb1JGWZlbBTCokC+YwhXMz+pHu0fu9a9hiWKB3iSJDd2YlFC3/EFL9jCbwYjAmvGzGDq8lb8NTAQbozdJYoHepIS6F2H/yK30Yz6jujzHEzdmaF50sEevChJu8Spj4qPRoRsRmSoiG0Rkaci2ziIyV0RWBO47hbw2VkRWisinInJ2ohpuMkvoRdhzeZEbeYiHuZ4nN/3Cv0YlWseOsG+fWx3LmASKZIx+GnBOnW23AW+qam/gzcBzROQoYAhwdOCYiSKSRrVjjV+CF2HzKWcal7GQE7mF+zIqyaaeYBmEr7/2tx0m4zUa6FV1HrC5zubzgNLA41Lg/JDts1R1t6p+AazErcFpTINKSqBD273MYghZVDOIObTIaZ0xc6PC6hT4Q9gCvUmwaLNuuqnqOoDA/UGB7b2AL0P2qwhsM6ZBRUXw7x/dzim8x0gmU1VwWE21ykxlgd4kSbzTK8NdUQpbL1RErhKRBSKyINMqzZkovPIKR706HkaPZo5exKpVtYN8kxYKTxcW6E2SRBvo14tID4DA/YbA9grg4JD98oC14U6gqpNUta+q9u3atWuUzTAZ4csv4dJLoU8feOCBei8HUy/Ly12CSnCh8LQP9hboTZJEG+hfAoYHHg8HXgzZPkREWovIIUBv4IPYmmgy2t69MHQo7NnjqlOGWVwhXP2bHTvc9rRmgd4kSaN59CIyEzgd6CIiFcA44F5gjohcAawGLgJQ1WUiMgf4D7APuFZVqxLUdpMJxo2Dd991SwMefnjYXbxWnmpoRaq00KGDy5+32bEmwSLJuhmqqj1UtaWq5qnqFFWtVNUBqto7cL85ZP8SVT1MVb+rqq8ltvkm3YSOtQ/v9jrccw8rfjSSwrFDPcffvVIs0z71MivLpVhaj94kmNW6MUkTOtbeQ9cwfsMwPuZY+r4zocHx95ISt9JUqHorT6WrTp0s0JuEs0BvkqKsDIYPd2Pr2exjBheTww4uYg5b97attW/o+HtZWc0YfXZg6l2thcLTXadOsLnuNBVj4stq3ZiEC/bkqwJXa8ZxJ6cxj2E8zaccEfaY1avrFzqrqqrpyWdEkAfo0gU2bfK7FSbDWY/eJNyYMTXBegBvUEwJUxnBdIZ5HpOfn8HZNqEs0JsksEBvEqqszK37CtCddZRRxHKO5HoeAdyqgK3qVCAO9tozNtsmVNeuFuhNwlmgNwkV7H1nUUUZRRzANgYxhx20IzsbnnoKpk514+4itcffMzbbJlSXLrBtG+ze7XdLTAazMXqTUMHe92/4PWfwFiOYyn84GoDS0pqx9nBj7iUltcfoIYOybYK6dHH3lZXQs6e/bTEZy3r0JqHy8+F03mIcd/I0w5jGZQDk5jZ+QbWoyPXuw/X2M0Yw0Fu9J5NAFuhN3IQrPDb49PXM4GI+5btcw0RAyMmBCRMiO2dREaxaBdXV1Ct0Fg8iki0iH4rIy4HnyV1UJxjobZzeJJAFehMX4QqPXX5ZNWc+PYyObGEws/mW9oi4fPoU6pWPAZaHPE/uojrBgn4W6E0CWaA3cRGaQhl08757+LHO5X95mI85DnC/BF591YcGhiEiecBPgckhm5O7qE5urrsPpiYZkwAW6E3MQlMog37IPO7it8xgKJO5stZrKZQe+RDwa6A6ZFvMi+o0aa0Fq2BpksACvYlIQwt/jBlTe98ubGQmQ/kvhzGKJ6i7Hk3nzjXn6tLF3ZK9oIiI/AzYoKoLIz0kzLawi+o0aa2FVq1cKpGVQTAJZOmVplF1SxEEC48FhfbmhWpKGU4ulfyUV9jT6gDYU/N6y5YubTx4TOixoedNwhj+qcC5IjIQaAN0EJHpBBbVUdV10S6q02RW2MwkmPXoTaMaKkVQtxzBLdzHQF7jVzxARW6fepOhOnRwa4x4SVaJA1UdGyi7XYi7yPoPVb0EPxbVsUBvEsx69KZRkZYi+D7vUkIxc7iIxxkNlS5ohxYhy4qga+HzGH7yF9WxQG8SzAK9aVR+vhtWCbcd3GudqWQWQyingJE8SXBIu+5wjNe5wp03WVT1n8A/A48rgQEe+5UA8Z+X26mTmyRgTILY0I1pVEMLf5SUQE5bZRqXcRAbGMQctnJgrX1Dh2NKStwwjpeMK3EQCevRmwSzQG8a1VApgqIimHfBA/ycl7mF8SzipLDnCA7HFBXB1VeHD/a5uRlY4iASFuhNglmgNxHxLEXw3nucNOc2+MUveLj6OgoKwh8fOhwzcSI880ztXxzTp7vJoc0uyIPLN92+Hfbu9bslJkNZoDfR+/prGDIE8vJgyhQQiXh910TXsEkrwUlTW7b42w6TsSzQm+iowuWXw5o1MHs2dOwI1AzzBGf2A7Rt63EO4xwYuKbxzTf+tsNkLMu6MdF5+GH461/h/vuhX/2SLzt31jyurEzqRKj0Y4HeJJj16E3TLVhA1U23MLftz8m66cZ6pQuaxVqv8WSB3iSY9ehN02zZwraBg/i6ugeDd05DkXq58s1irdd4skBvEsx69CZyqqw+60rabPySwTqLr+m8/6XQHnuzWOs1njp0cPcW6E2CWKA3EZs/YiL585/jdv7Ae5xS7/Vgjz3SzBsTYD16k2AW6E1kFi3iuNJf8QoDuZ+bwu4S7LE3i7Ve48l69CbBYhqjF5EbgStxdbk/BkYAOcBsoBBYBQxSVZv2l862boXBg9lIV4ZTiobpH9TtsQdnzZoItGzpfoAW6E2CRN2jF5FewP8CfVX1GCAbV+417JqbJk2pwqhR8MUX3NhtJpV0qbdLdrb12GN24IEW6E3CxDp00wJoKyItcD35tXivuWlSQEMrRYXb5/Yuk2DWLLj7bs6//4f1xt5FoKrKXYhN1upQGenAA91fTsYkQNRDN6q6RkTG42p27wT+rqp/F5Faa26KyEHhjheRq4CrAPItHSMpGlopKtgbD93nOJbw281jeCPrLNbn3bp/n+Jid6yI6/B7ncs0gfXoTQLFMnTTCdd7PwToCbQTkUsiPb5J62qauPCayDRmTE0Pfvhwt60925jDIDbTmYurn6H4jizKytw5Vq92wzWq9c9lk6KiZIHeJFAsF2N/DHyhqhsBROR54Pt4r7lpfOY1Yamysmbt1qoqAOVxRvMdVjKAN9nIQVBe+6+BKo+1lWxSVJQOPNB+eCZhYhmjXw18T0RyRERwq/Isx3vNTeOzSEfILmcql1DG7/gd/+L0/dvr/jUQy3uYOqxHbxIo6kCvqu8DzwKLcKmVWcAk3JqbZ4rICuDMwHOTAsJNZKrraJbyCNfzBgP4A7c36fw2KSoGFuhNAsWUR6+q44BxdTbvxmPNTeOv4EXS4cPDD73k8C1zGMRWOjA8azrV1dmNnjM729WUz8+vvQi4aaIOHdyfTHv3urx6Y+LIZsY2M0VF3uPrj3EtR/AJV7aezp+e7t7g2q7gevClpbZ4SFwEyyBYiqVJAAv0zUxZWfj1Wi+llMso5ZEDf8PQKT+mqKjh8XYraxBnVu/GJJAF+mamuLh+WuQRLGci17D+iNPo+sg4iotdquWmTfWPz8lx67taDz7OLNCbBLJ69M1M3Qy+tuxgDoP4lna8e+0MRl6dvT+75ttva+8r4sb3LcAngAV6k0AW6JuZ/Hw3izVoAmM4lqVcetDrzBvfs8EUSlV49dXEt7FZskBvEsiGbpqZ0BTLocxgJJO5r8VYzn7g7Ijm69icngSxQG8SyAJ9BgstTtali7sNGwZt20K/jp/xBKP4oPUP6DXlrkYvvgbZhKgEsawbk0AW6DNUsDhZebkbcgmWOVCF7ZW7ePKbQbRs35p+K2dy8aVuBK+xCVU2ISqBgoF+82Z/22EykgX6NOdVdjhcAbOgB7mR43QJo9uWUvavvP3HFxe7i63BlaFyc93NVolKglat3A977Vq/W2IykF2MTWMNlR32GksfxGxG82f+xC1M2/hT5tQ5/s9/dr3+ggKb6Zp0eXmwZo3frTAZyHr0acyr7HBxcfix9MNYyZOM5P84hWJKyM6uf3zd+vK2mEgS9eplgd4khAX6NObVa1+9uv54eyt2M5vB7KMFQ5hFq5yWnqUQgjK9vryItBGRD0RkiYgsE5E7A9s7i8hcEVkRuO8UcsxYEVkpIp+KyNlxbZAFepMgFujTmFcGTH6+G3KZNKlmvP3RNjdzEosYwTSyCvL3v9aYDE+n3A2coarHA32Ac0Tke3iseywiR+HWRT4aOAeYKCKNV36LVK9esGED7NkTt1MaAxbo05pXlkx5uUulBFeq4J/XP8fIXY/yIDeypODc/WPvkZQtzuR0SnW2B562DNwU73WPzwNmqepuVf0CWAn0i1uDevVy9+vWxe2UxoBdjE1rwQulY8bUrBAVVFkJl18On7z2BTeVXcEHnMyt3MveMGu7hlsDFppHOmWgR74Q+A7wmKq+38C6x72A90IOrwhsq3vO6NZDDgb6NWsi+3PLmAhZjz7NFRVB+/bhX9M9ezi3bDAAg5nNXloBtcfei4pcr18VnnmmZqinuaRTqmqVqvYB8oB+InJMA7uHK9ys9TZEux5yaKA3Jo6sR58BvMbR/8itnMx8LuB5VnFIo8cUFWV+YPeiqltE5J+4sXevdY8rgINDDssD4pf4boHeJIj16DNAuNGBc3mRG3mIR+R6/soFER3T3IhIVxHpGHjcFrfg/Sd4r3v8EjBERFqLyCFAb+CDuDWoc2do3doCvYk7C/QZoKSk9upz+ZQzjctYKCexYuR99S64Noex9wj1AN4SkY+A+cBcVX0Zj3WPVXUZMAf4D/A6cK2qNpKk2gQilmJpEsKGbjJA6EXZbyr3MoshZEs1FeNn8/CvWvM//d2Y/OrVtrZrKFX9CDghzPZKPNY9VtUSIHG/JvPyoKIiYac3zZP16DNEUZFbEWrvzbdzCu/RYdaTnPerw/a/tmqVre2aFqxHbxLAAn0aq1vQ7K2bX4Hx42H0aBg0yO/mmWj06AHr1/vdCpNhLNCnqbpliKvKv+S4+y9lc/7xzDz5gbAVLU0a6N7dreG4fXvj+xoTIRujT1OhBc2y2cdMhtKKPQzY+heWXdcmbEVLG7JJA927u/t166B3b3/bYjKG9ejTVOi6r3dzBz/gXa5iEvO39A5b0XLMmOS2z0QpGOi/+srfdpiMYoE+DZWVuUw8gLN5nbHcyyRGMouhnsdUVtoQTlqwQG8SwAJ9GiouduPyPVjLMwzjI45lDBP2rwrV0HEmxVmgNwkQU6AXkY4i8qyIfCIiy0XklIZqeZv4WL26Zly+LTsZxBx20RZVmDCh4eNMisvNhexsC/QmrmLt0U8AXlfVI4DjgeV41PI28ZOfD+O4k9OYx2ge51OOAFwhsqIi7169lT1IA1lZ0K2bBXoTV1EHehHpAPQHpgCo6h5V3YJ3LW8TJ1OGvkExJUxlBNMZBtQuazBhQv0681b2II10726B3sRVLD36Q4GNwFMi8qGITBaRdkCtWt7AQeEOFpGrRGSBiCzYuHFjDM1Ib3UnPTV6wfSrrxgwtYitvY7kvoMfCVtSuO7qUs2l5HDGsEBv4iyWPPoWwInA9YHFGibQhGEaVZ0ETALo27dvvZrezUFw0lPEOe9VVXDxxbBtGx3n/4PlR7fzPHdzLjmc9nr0gMWL/W6FySCx9OgrgApVfT/w/Flc4F8fqOFNnVrepo7QSU9BwZz3sL383/8e3noLHnsMjj46ya01SdO9uyuDUF3td0tMhog60KvqV8CXIvLdwKYBuPKtXrW8TR1eWTCVlTWlDYK9/DeK34I774Rhw+Cyy5LaTpNk3bu7v97qrg9pTJRiLYFwPVAmIq2Az4ERuF8ec0TkCmA1cFGM75Gx8vNrz3D10m7HBo7748Xw3e/CxIk1s6VMZgrNpW/KUoTGeIgpvVJVFwfWxjxOVc9X1a9VtVJVB6hq78D95ng1NtOUlNTPjqlLqOYZhnFA1RZeuXS29wKxJnOE1rsxJg5sZqyPgtkxDc1mvY17OZu/M4YJDPr9cVbGoDno0cPdW6A3cWKB3mdFRd6d9B/wNndzBzMZwpOMZMcOK2PQLAQD/dr4rTtumjcL9Ckg3EXZXDYxk6F8zqGM4glAPPc1GSYnBzp0sB69iRsL9EnkNTmqc+fa+wnVPM2ldGUjg5jDNjrUfl3crUsXq0iZsXr0sEBv4sYWHkkSr8lR774LW7fW3vdmxjOQ17iGx1hcZ+1qDZlaVlkJl1/uHtvkqAzTs6cFehM31qNPEq/JUZMmwd69NdtO4f/4A7fzF37J44xu9Lx79ti4fUbq0cPG6E3cWKBPEq+x9aqqmsedqWQWQyingCuZTHBcPtpzmzTWs6cL9Nosq4OYOLNAnyReJYKzs4OPlGlcRjfWM4g5bOXAmM9t0lh+PuzeDRusgoiJnQX6JAk3OSonx43T5+TAr3iAn/MyNzOeRZwU8XlbtbLywxmpoMDdRzJ12phGWKCPM6/MGq/SwRMnwnO/fp97uY3n+AUzO19Hbq7bp6a3H15uLkydahdiM5IFehNHlnUTRw2VHQZ30XT1avdXeUlJIEB//TXnPDUYCvO4cNFkLuwktc43bFj4YdqCAli1KqEfx/jJAr2JIwv0cdRQ2eGdO8P8AlCl6LnL3UW3d96BTrWX1y0qgksuCf9edgE2w3Xs6CZNWaA3cWCBPo4aKjtc144dsOJ/H4Gv/woPPAD9+oU9tqAg/P91uwDbDHj94xvTRDZGH0dNCb59mc/tX98MP/853HCD535eF3HtAmwzYIHexIkF+jjyCsp1q1MeyBZmM5iN2d1h2rQG68vb+q/NmAV6EycW6OPIKyhPmBD6C0B5kpHks5qlxbPqF7rxOO+qVW5luVWrLMjHi4gcLCJvichyEVkmImMC2zuLyFwRWRG47xRyzFgRWSkin4rI2QltYEEBfPONuxkTAwv0cRYMys88454PG+Yu0g4f7v7fXsPjXMSzvPS9exhV+v3668KaZNoH3KSqRwLfA64VkaNwi9y/qaq9gTcDzwm8NgQ4GjgHmCgijSTBxsAyb0ycWKBPgGCaZei6r6WlMHHkhzzW6kbWHP8TLl1yU711YS3YJ5eqrlPVRYHH24DlQC/gPKA0sFspcH7g8XnALFXdrapfACuB8FfR4yEY6C2P1sTIAn0MvCZHhUuzzN6xlSN+Owi6duVnlU/z7c7aP3pbVMRfIlIInAC8D3RT1XXgfhkABwV26wV8GXJYRWBb3XNdJSILRGTBxo0bo2+U9ehNnFh6ZZQamhxVP81SeYJRFFR/zt9H/JPFv+8S9pyWG+8PEWkPPAfcoKpbxfvieLgX6k1nU9VJwCSAvn37Rl+VrFs3aNPGAr2JmfXoo+Q1Oaq4uH6a5UieZCizuIO7ufjxH3om2VhufPKJSEtckC9T1ecDm9eLSI/A6z2AYGWxCuDgkMPzgMTVEhZxXwoL9CZGFuij5NX7Xr26do77sXzEBMbwN87iXm6jsjJ8SQMRy41PNnFd9ynAclV9IOSll4DhgcfDgRdDtg8RkdYicgjQG/ggoY20FEsTBxboo+TV+87Pd5k3ubnQju3MYRBf04lhPIM28ONWtbRJH5wKDAPOEJHFgdtA4F7gTBFZAZwZeI6qLgPmAP8BXgeuVdWq8KeOEwv0Jg5sjD5KJSW1x+gBWraE7dvdxdnOnZQnskbTu3oFP+YNNnIQOTnQtm34kgjB624meVT1HbxXdxngcUwJkLy/vQoLXU367duhffukva3JLNajj1LdyVHB0sLBoZlzNz9FUfV07ms7jn/JjzwmTzlW0sB4OvZYd79kib/tMGnNAn0EGqoxH5yx2r69W78V4CiW8SjX8SZnMKlrca0ZrVbSwDTJSYFFaBYu9LcdJq3Z0E0jGkqjDA3OwYuzOXzLX7iIrXSgiDI2fFl/4mQw4BvTqJ49oXt3C/QmJjH36EUkW0Q+FJGXA88964Sko4bSKEMFL84+ynUcwSdcwnTW091SJk1sROD442HZMr9bYtJYPIZuxuCmjgeFrROSrhpKowxVUgIjW5Uygmn8nt/wJj+2sXcTH4cfDp99Fj4v15gIxBToRSQP+CkwOWSzV52QtBIcl/f6v6VaZ7z+xOVMlGt4r/Vp3MU4G3s38dO7N2zb5rJvjIlCrD36h4BfA9Uh27zqhNQSt3ogCRBalKwhwfH6WVN3sOWsQWzZ244Ld88gryC7Zk1YvC/mGhORww9395995m87TNqK+mKsiPwM2KCqC0Xk9KYeH7d6IAkQblzey44dsPeaMXTcvZSzeZ219IQ6i4JHcjHXGE+9exG8NAcAABKYSURBVLv7FSvghz/0ty0mLcXSoz8VOFdEVgGzcLMLp+NdJySlhfa6mzIR8WLKGLZ7Mn9gLH+nZh2K4AXbSC/mGuOpoMDNxrMevYlS1IFeVceqap6qFuIWY/iHql6Cd52QlFW3fryX7DqZkr35jCcYxdv8gN9yV739V6+O/GKuMZ6ys+Gww1yP3pgoJGLCVNg6IakskqGanBz3yyA4q7U1u5jDIHbRhms6zqQqzChYfn7DNXGMiVgw88aYKMQl0KvqP1X1Z4HHlao6QFV7B+43x+M9Eqmh3nXo7NWJE2tmtT7EjfRhCR/fVMptj+Z5ljXwWjDc0i5Nk/TuDStXumnYxjSRlUDAu3ddUFB77dfCQvd41R9nczV/hltuYe0JP93/F0FwaCc0tdJKHpi4OPxw2LULKir8bolJQxbo8e51DxxYf+3Xe69cyd7LRsIppzDj6JJaaZhVVTW99dBAHloTJ1jzxpgmCc28MaaJLNDj3et+9dXaY/et2M20XYP5dk8LmDWL28e1tIwakxyWS29iYIE+IFyvu+7Y/Xhu5iQWMbx6GuTnW0aNSZ6ePd2fi9ajN1Fo1oG+sRmroWP3v+A5rudRHuBGlhScW+91r+OMiQsRN3xjPXoThWYb6OvmzgdnrIYG++DY/SF8zhSu4H36MZZ7KS93vxgGDrSMGpNEhx8On3zidytMGmq2gT6SGatFRTB54h6ebzUEgKHMYg+tAPeLobQUhg+3jBqTJMccA59/Dt9+63dLTJpptguPRDq+PnTxrbBnPqO6PscXGw+p9dqOHe6C7apViWmjMbUce6z783PZMujXz+/WmDTSbHv0EY2vv/giPPQQXHcdT276Rdj97cKrSZrg+rFLl/rbDpN2mlWgD734un07tGpV+/Va4+vl5TBiBJx4Iowf7/mLISvLyg6bJDn0UGjbFj7+2O+WmDTTbAJ93YuvlZXuPjc3zPj63r0wZAjs2wdz5kDr1mEnVYGbJFX3Iq4xCZGVBUcfbYHeNFnGB/pgL/6SS+pffN27F9q3DzNj9fbb4b33YPJkVzWQmklVdStYgk2SMkl07LEW6E2TZXSgj2SlqHpj7K+8AuPHw9VXw6BBtV4qKvKuKWVj9SYpjj3WLSloywqaJsjoQB9J+eFaY+9ffgmXXgrHHw8PPtj4/hFsNyaujjvO3S9Z4m87TFrJ6EDfWC+71sXXvXth6FDYs8eNy7dpE/YYKztsfNWnj7v/8EN/22HSSkYH+oZ62fUmN40bB+++C088UVNAKgwrO2x8lZvrvtgW6E0TZHSg9+p9T5/uXisudokMw7u9DvfcA1deCRdf3Oh5reyw8dUJJ1igN02S0YHeq/cNNRdpe+gaxm8YxjI5hlmnTPC3wcZE4oQTXHGz7dv9bolJExlfAiG4ylOowsLAilDsYwYXk8MOfqlz2HlXDkMu96WZxkTuhBPcJJCPPoLvf9/v1pg0kNE9ei/Bi7S/5S5OYx6jeZxPONJSJJsZEZkqIhtEZGnIts4iMldEVgTuO4W8NlZEVorIpyJytj+txs3WBhu+MRFrloE+Px8G8Aa/4fc8xWU8w6X7t0PjdepNxpgGnFNn223Am6raG3gz8BwROQoYAhwdOGaiiISZPpcEvXpB166wYIEvb2/ST7MM9Pff8hVlXMJyjuQ6HgVqUiQjqVNvMoOqzgM219l8HlAaeFwKnB+yfZaq7lbVL4CVgD8lJEXg1FPh7bd9eXuTfppfoK+q4sLni8httZUbesxhp7QjN9fViho2zM2XsnVgm7VuqroOIHB/UGB7L+DLkP0qAtvqEZGrRGSBiCzYuHFjYlp52mnw3//CmjWJOb/JKM0v0JeUwD/+QYvHH2Xu2qN55hnYubOmyJmVODAeJMw2Dbejqk5S1b6q2rdr166Jac2PfuTuX345Mec3GaV5Bfp//hPuvNNVOBsxAoisTAJYiYNmZL2I9AAI3AeLylQAB4fslwesTXLbahx3nKtkOW2ab00w6aP5BPr1612Jg9694fHH3TgnkffUrcRBs/ESMDzweDjwYsj2ISLSWkQOAXoDH/jQPkfEdVjeew+++sq3Zpj00DwCfXW1G4DfssXVsWnffv9LkfTUc3Nt9msmEpGZwL+B74pIhYhcAdwLnCkiK4AzA89R1WXAHOA/wOvAtapa5U/LA844w93/61++NsOkvqgDvYgcLCJvichyEVkmImMC2z3zkH1z770wdy5MmFBT/S/Aa0GRoJwcd5jJPKo6VFV7qGpLVc1T1SmqWqmqA1S1d+B+c8j+Jap6mKp+V1Vf87PtgMunb98e3nrL75aYFBdLj34fcJOqHgl8D7g2kGscNg/ZN2+/DXfc4YZtRo6s93LdMgm5uR6rThmTalq0gLPOghdecKuhGeMh6kCvqutUdVHg8TZgOS7dzCsPOS6aNJlp0yYX4A87zFWlDIzL1z0H1BQp27TJ3axgmUkLl1ziFiGZO9fvlpgUFpcxehEpBE4A3sc7DzlmTZrMVF3tkuI3bXLj8gcc0PRzGJPqBg6ETp1cSVZjPMQc6EWkPfAccIOqbm3CcU2eVBIuFbLuZKZgb/3W7PHw2mt8MPRB6NOnwbVjbUKUSVutW8PgwW74Zts2v1tjUlRMgV5EWuKCfJmqPh/Y7JWHXEs0k0q8UiGD24O99Z7l/0cJt/MXfsmPZl/NNddEsXasMeli2DA36+/55xvf1zRLsWTdCDAFWK6qD4S85JWHHDOvVMjOnd19cTG02VHJLIZQTgFXMpkdO4XHH2/i2rHGpJNTToFDD7XhG+Mplh79qcAw4AwRWRy4DcQjDzkeSkqgZcv627dtc7351eXKU4ygG+sZxBy2cmBE57U1X01aC06eevNNq31jwool6+YdVRVVPU5V+wRurzaUhxyroiLo0KH+9j17XG++OOdBzuX/cTPjWcRJEZ3TUihNRhg2zGUXPP203y0xKSitZsaWlbniY+F0K3+f3+64lee5gEe5rtFzBdeOtRRKkxG+8x0480x45BHYvdvv1pgUkzaBPnihNZyOfM1sBlNBHlcwhfCFBiE72yZCmQx2002wbh0895zfLTEpJuUCvdeEKO8qk8oUrqAXaxjMbLYQvuKCiPtFYROhTMY680w3OfDPf/a7JSbFpFSgb2gyk1f64/U8wi94gVv5I/MbWPBHFUpLbWKUyWBZWTBqlCv7sXRp4/ubZiOlAr3XhKgxY9x3uK6TWMB4buYlfs6D3Njo+W1ilMl4I0a4SVQTJ/rdEpNCUirQe/XaKyuhqk5B2A58w2wG8xXduYxpeI3LR/oexmSELl3cuOS0aa78hzGkWKCPfNKSMpkrKaCcIcziazrX2yM7O9b3MCZN3XSTyzm+4w6/W2JSREoF+sZqwweN5nEu4lnuyPoD/+b7Yfepqqp/LpsYZZqFo46Ca691qWUrV/rdGpMCUirQ160NX1DgasOH6sOHPMiNvNXmJxwz7eawY/dQk0IZei5LqTTNxtixbhr52LEuE8E0aykV6MEF4mBt+FWr3OpOwZ75AWxlDoPYJF2pfPBpioZl8fTT3j33uueyIG+aje7dYdw4ePZZW0DcpF6gr2t/Lz9fmcRVHMIXLCueyS+v7lL7deu5G1PbrbdC//7wq1+5iVSm2UqJQL95c8OrRhUVwariJxnCbFqU3MVZd/+w/uurrOduTC1ZWfDkk7Brl0u73LvX7xYZn6REoC8vb2TFp48+csn0Z54Jt/m7BK0xaeXww+Ghh+Bvf3MXaf/7X79bZHyQEoG+urr289CJTbOnbOe/Jw1i7a5OnLR8OmUzU6LJxqSPUaPcWP3mzfD978OLcVsiwqSJlI2aq1dD2XSletRoCvet4GJmsKjiIFvf1ZhoXHghzJsHPXrA+efDyJH1ZyGajJWygT4/Hxbf8BRDq6ZzJ+P4F6cDVsbAmKgdfTR88IG7SDt5MlxzjQX7ZiIlAn3dXPicHHh09DLurLyONzmDEmpHditjYEyUWrWCe+91wX7SJOjd2/X0TUZLiUBfUFA7PXLKw9/ys9KL+DarA0WUUU3tegZWxsCYGN1zjxu3b9kSzj4b7r/fZUKYjJQSgb5z59rpkUPeuQ4++YSPfj2dbTnda+1rZQyMiQMRN27/7rtw6qlw881ugfEzznCzFL/80u8Wmjhq4XcD6nn6aTeT7447GHDXj5l0jBuTX73a9eSDM16NMXHQpQvMnQuffebW1nz+ebjhBnc78kg4+GDo2dPdevSo/bhTJ2jbFlq0cL84TMoSTYE6GH379tUFCxbAJ5/ASSfBySe7Fe29SlAa00QislBV+yb7ffd/t9PJZ5+5gP/ee25G7dq17t7rwm1Wlhv7z86uf8vKqn8Lbq/7y8Hrl0VojAo+DrctksfRHufne+fkQEUF4UT6vU6dHv2OHTBokPtQZWUW5I3xy+GH15+YWF0NGzfWBP61a2HrVvj2W7cY+e7dbp+qqppbdXX9W3B73V8adTucqrUDf7jHjb0e7+P8eu9WrYhV6gT6MWPg44/htdegVy/KymzIxqQeETkHmABkA5NV9V6fm5QcWVnQrZu79enjd2tME6VGoN+82eX13nYbnHPO/rVjg8sKBssigAV74x8RyQYeA84EKoD5IvKSqv7H35YZ07CUyLqhvNxd+b/7bsB77VibKGV81g9Yqaqfq+oeYBZwns9tMqZRqRHos7Jg1ix39R7vCVE2Ucr4rBcQmndYEdhmTEpLiUC/eN8xFP4gb38NG68JUTZRyvgsXFpIvbQ1EblKRBaIyIKNGzcmoVnGNCwlAn0V2bXKE4dbO9YmSpkUUAEcHPI8D1hbdydVnaSqfVW1b9euXZPWOGO8JCzQi8g5IvKpiKwUkYiKyIeOw7dtW7M9N9dWjTIpYT7QW0QOEZFWwBDgJZ/bZEyjEpJ1E0t2QrBnH3oxdufORLTSmKZR1X0ich3wN1x65VRVXeZzs4xpVKJ69FFnJ2RnW8aNSV2q+qqqHq6qh6mqDSaatJCoPPpw2Qn/E7qDiFwFBLLjWwNuFq/XLOvychBZuDDeDU2SLsAmvxuRIOny2Qr8eNOFCxduEhGvspCp9LNLlbakSjsgPdoS0fc6UYG+0ewEVZ0ETAIQkQWqC5JehyRZ3OdLfp2VZMjkzxYPqup5NTaVfnap0pZUaQdkVlsSNXQTUXaCMcaYxEtUoLfsBGOMSREJGbqJIjthUiLakUIy+fNl8mdLtFT62aVKW1KlHZBBbUmJevTGGGMSJyVmxhpjjEkcC/TGGJPhfA/00ZRKSFUicrCIvCUiy0VkmYiMCWzvLCJzRWRF4L6T322Nlohki8iHIvJy4HnGfLZk8fs7LyKrRORjEVksIgsC25Ly7ygiU0Vkg4gsDdnm+d4iMjbwc/pURM5OQlt+JyJrAj+bxSIyMNFtiSZuNLktqurbDXeh9r/AoUArYAlwlJ9tivHz9ABODDw+APgMOAr4E3BbYPttwB/9bmsMn/FXwAzg5cDzjPlsSfr5+f6dB1YBXepsS8q/I9AfOBFY2th7B/7vLMHNqDwk8HPLTnBbfgfcHGbfhLWlqXEjmrb43aPPqIUcVHWdqi4KPN4GLMfNEj4PKA3sVgqc708LYyMiecBPgckhmzPisyVRqn7nk/LvqKrzgM0Rvvd5wCxV3a2qXwArcT+/RLbFS8LaEkXcaHJb/A70GbuQg4gUAicA7wPdVHUduH9U4CD/WhaTh4BfA9Uh2zLlsyVLKnznFfi7iCwMlCIBf/8dvd7br5/VdSLyUWBoJzhckpS2RBg3mtwWvwN9RAs5pBsRaQ88B9ygqlv9bk88iMjPgA2qmq71hlJFKnznT1XVE4GfANeKSP8kv3+k/PhZPQ4cBvQB1gH3J6stTYgbTW6L34E+40oliEhL3D9Wmao+H9i8XkR6BF7vAWzwq30xOBU4V0RW4YYbzhCR6WTGZ0sm37/zqro2cL8BeAH3Z7+f/45e7530n5WqrlfVKlWtBp6kZkgkoW1pYtxoclv8DvQZVSpBRASYAixX1QdCXnoJGB54PBx4Mdlti5WqjlXVPFUtxP07/UNVLyEDPluS+fqdF5F2InJA8DFwFrAUf/8dvd77JWCIiLQWkUOA3sAHiWxIMLAGXID72SS0LVHEjaa3JRFX1pt4xXkg7irzf4Fiv9sT42f5Ae5PqI+AxYHbQCAXeBNYEbjv7HdbY/ycp1OTdZNRny1JPz/fvvO4bJ8lgduy4Psn698RmIkbEtmL65le0dB7A8WBn9OnwE+S0JZngI8D/4dfAnokui3RxI2mtsVKIBhjTIbze+jGGGNMglmgN8aYDGeB3hhjMpwFemOMyXAW6I0xJsNZoDfGmAxngd4YYzLc/wc7eUraJcG0QQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "train(10, new_X, off_Y, 200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 逻辑回归"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAcZElEQVR4nO3df5Bdd1nH8ffTpEi3aJqkmxgLuwtjpoCtLWTHqVSZylLlR6HVmWqZLbPjIPsPg42j48TZPyp/7Ig/Rssf4kwEJWPWaoViY/9gyKwUhxmpbqG0xRCDsAnFpVmSEtR0oCWPf9yz9GZzf5x77/nx/X7P5zWzc/eevT+ec2/69Hu+z3O+x9wdERGJz2V1ByAiIsNRAhcRiZQSuIhIpJTARUQipQQuIhKprVW+2dVXX+1TU1NVvqWISPQee+yxb7v7+ObtuRK4md0DvBcw4C/d/T4z2wH8PTAFrAK/6u7P9nqdqakpVlZWBgxdRKTZzOxkp+19p1DM7DpayftngBuA28xsL3AAWHb3vcBydl9ERCqSZw78NcDn3f28u78AfBb4ZeB24FD2mEPAHeWEKCIineRJ4E8BbzSznWY2BrwNeAWw293XALLbXZ2ebGbzZrZiZivr6+tFxS0i0nh9E7i7HwP+EDgKfAr4EvBC3jdw94PuPu3u0+Pjl8zBi4jIkHK1Ebr7R9399e7+RuAscAJ4xsz2AGS3p8sLU0RENsuVwM1sV3Y7AfwKcD9wBJjLHjIHPFRGgCIi0lnePvBPmNlO4Hngfe7+rJl9EHjAzN4DnALuLCtIERG5VK4E7u4/32HbGWCm8IhEREqw9OQSC8sLnDp3ioltEyzOLDJ7/WzdYY2k0jMxRUTqsPTkEvP/NM/5588DcPLcSeb/aR4g6iSutVBEJHkLyws/TN4bzj9/noXlhZoiKoYSuIiUaunJJabum+KyD1zG1H1TLD25VHkMp86dGmh7LJTARaQ0G1MXJ8+dxPEfTl1UncQntk0MtD0WSuAiUppQpi4WZxYZu3zsom1jl4+xOLNYaRxFUwIXkdKEMnUxe/0sB99xkMltkxjG5LZJDr7jYNQFTFAXioiUaGLbBCfPXboSah1TF7PXz0afsDfTCFwkcXUWEVOdugiFErhIwuouIqY6dREKc/fK3mx6etp1RR6R6kzdN9VxCmNy2ySr+1erDyhidZ7JaWaPufv05u2aAxdJWChFxNiFeianplBEEpZq/3PVQmmH3EwJXCRhKiIWI9QjGSVwkYSpiFiMUI9kNAcukrgU+5+rtjizeNEcOIRxJKMRuMiIQlisScoV6pGM2ghFRrC5OwFaI7MQ/uOWdHRrI8x7TczfMrMvm9lTZna/mb3UzHaY2VEzO5Hdbi8+bJGwhdqdIPnFfATVN4Gb2TXAbwLT7n4dsAW4CzgALLv7XmA5uy/SKKF2J0g+dZ+pOqq8c+BbgSvMbCswBvw3cDtwKPv7IeCO4sMTCVuo3QmST+xHUH0TuLt/E/gTWleeXwPOufungd3uvpY9Zg3Y1en5ZjZvZitmtrK+vl5c5CIBUJ91Meqaxoj9CCrPFMp2WqPtVwI/AVxpZnfnfQN3P+ju0+4+PT4+PnykIgEKtTshJnVOY8R+BJVnCuXNwNfdfd3dnwceBN4APGNmewCy29PlhSkSrtnrZ1ndv8qFey+wun+1sOQdc3FtEHVOY8R+BJUngZ8CbjKzMTMzYAY4BhwB5rLHzAEPlROiSPPEXlwbRJ3TGLEfQeXqAzezDwC/BrwAfBH4DeBlwAPABK0kf6e7n+31OuoDF8mnScvANmlfhzVSH7i73+vur3b369z93e7+PXc/4+4z7r43u+2ZvEUkv9iLa4OIfRqjTjqVXiRAsRfXBhH7NEadlMBFKpanONm0UWlZheDUKYGLVChvcVKjUslDi1mJVEgFOxnGSEVMESlGk4qTVWpKz/xmSuAiFWpScbIqTeqZ30wJXKRCTStOViH2BalGoQQuUiEVJ4vX5GkpXRNTpGK6RmWxJrZNdCwMN2FaSiNwEYlak6ellMBFJGpNnpZSH7iISODUBy4il4ipfzqmWKuiIqZIQ230T2+04G30TwPBTT/EFGuVNAIXaaiY+qdjirVKSuAiDRVT/3RMsVZJCVykoWI6rT+mWKuU56r015rZ420/3zWz/Wa2w8yOmtmJ7HZ7FQGLSDFi6p8OLdZQCqp9E7i7H3f3G939RmAfcB74JHAAWHb3vcBydl9EIhFT/3RIsYa0eNZAfeBm9ovAve5+s5kdB25x9zUz2wM84u7X9nq++sBFJHZ1rOleVB/4XcD92e+73X0NILvd1eWN581sxcxW1tfXB3w7EZGwhFRQzZ3AzewlwDuBfxjkDdz9oLtPu/v0+Pj4oPGJiAQlpILqICPwtwJfcPdnsvvPZFMnZLeniw5ORPILpbCWupAKqoMk8Hfx4vQJwBFgLvt9DnioqKBEZDAhFdZSF1JBNVcR08zGgG8Ar3L3c9m2ncADwARwCrjT3c/2eh0VMUXKoYslp61bETPXWijufh7YuWnbGWCmmPBEZBQhFdakOjoTUyQBIRXWpDpK4CIJKLOwpuJouJTARRJQVmFNxdGw6Yo8ItKViqNh0BV5RGRgwxRHNeVSHSVwEelq0OKoplyqpQQukqCiRsGDFkd15ZxqKYGLJKbIUfCgxVH1o1dLRUyRxNRZeFTRsxwqYopEoIipjzpHwSEt9NQESuAigShq6qPOszJDWuipCZTARQo0ygi6qAJg3aPg2etnWd2/yoV7L7C6f/Wi5J16i2HV+5drMSsR6W9jBL2RhDdG0ECuEWhRUx8b77WwvMCpc6eY2DbB4sxi7aPgUT+f0NWxfypiihRk1AJe6gVA7d/wVMQUGdCgh8OjjqDrnvooW+othnXsnxK4SAfDFBRHLR6mXgBMfcnbOvYvVwI3s6vM7ONm9hUzO2ZmP2tmO8zsqJmdyG63lxalSMWGKSgWMYLuVQCMXepHGHXsX94R+IeAT7n7q4EbgGPAAWDZ3fcCy9l9kSQMczic+gh6VKl/PnXsX98ippn9GPAlWtfD9Lbtx4Fb3H0tuyr9I+5+ba/XUhFTYpF6wU3iMkoR81XAOvDXZvZFM/uImV0J7Hb3NYDsdlehEYvUKPXD/WHF1sfdLd7Y9qObPCPwaeDzwM3u/qiZfQj4LvB+d7+q7XHPuvsl8+BmNg/MA0xMTOw7efLSUY1IiJaeXAqul7pOm/ucofU/tVCnQbrFO3fDHIe+dCia/YDuI/A8CfzHgc+7+1R2/+dpzXf/JJpCEWmM2KaVusW7xbbwA//BJdtD3Q8YYQrF3b8FfMPMNpLzDPAfwBFgLts2BzxUUKzRWVqCqSm47LLW7dIAR2OjPFekSiH1ceeZAukWV6fk3evxIct7Kv37gSUzewnwNeDXaSX/B8zsPcAp4M5yQgzb0hLMz8P57Gjs5MnWfYDZPkdjozxXpGoT2yY6jmir7uPOe8p6t3i7jcBj7EfP1Ubo7o+7+7S7/7S73+Huz7r7GXefcfe92e3ZsoMN0cLCiwl4w/nzre3DPvfuuzUal+r1G9VWVdjtF0e/Hv2N5588dxLDLol3ft98MgVqnYk5olNdjrq6bc/7mI3RuJK4VCHPmadV9DnniaPXVE778wEc/2ES34j3w2//cDL96FrMakRTU61ku9nkJKyuDvfcQV9HZFShFCjzxNHrMUAQ+1E0LWZVksVFGLv4aIyxsdb2YZ67Wb+RvIqgcQi977iOAmWnzyRPHL2mckIqtFZBCXxEs7Nw8GBrpGzWuj14MF8Rsv253Uz0qKtsFEFPngR3TbuEqsiLDJel6oWYun0mO67Y0TeOXlM5qS+YtZmmUAKxuSMFWqPzXv8zGGX6RqoTyvREL1WfpNPtM9l5xU6ee+G5oeOI7WSjvDSFUpFhpzSGGcmPUkCV6sRwWF/1Qkzd9v3sc2dHiiP1BbM20wi8QMOMokehEXgcYhiBV02fyWA0Aq/AKD3hwxilgNp07UdKV//CElcvlldgLLN/OvTiaDdaLKwYSuAFqnpKY5QCapNdVPy9bokzb5jnzAvlFRjLOqyPoTjaTdOmOsqiKZQCaUojDhd9T/un4Ko4D+U1DdEcmkKpQJOnNGLqR7/oiGhb+AXGbmIojkq5lMAL1NQpjdj60S/qrT8Xb99w03qe5VJK4AWbnW1Nl1y40LrtlLxjGq3mUXXxdlQXHSktL8L34yym1V0IrKKAGmuRFqqJPe9yslKQFJeQja0ffeNzXliAU0/NsmMn8OYFzr4Q15V3NmKs46pBeZd0Df09ylJV7CpiVizFQmeK+yS9VVFAjblIW3TsKmIGIrbRah4hFG9DnJaK+fC/nyoKqDEXaauKXQm8Yt0Wp+q1aFXo6i7ehlhEjblHO48qCqgxF2mrij1XAjezVTN70sweN7OVbNsOMztqZiey20uuSC+XCmG0WoY8xduyVFVEHWRE3e+qMbGrooBad5F2FFXFPsgI/Bfc/ca2eZgDwLK77wWWs/vSR92j1RRVMS016Ig65sP/PKo4kzLmszWrij1XEdPMVoFpd/9227bjwC3uvmZme4BH3P3abq8BKmJKOaooog5alIq5ACfhGbWI6cCnzewxM8ua3tjt7msA2e2uLm88b2YrZrayvr4+TOw/FGKhSuo3zLTUoAXGQUfUMR/+N1lshee8Cfxmd3898FbgfWb2xrxv4O4HsyvaT4+Pjw8VJIRZqJIwDDotNUyBcdCiVMyH/00VY+F54D5wM/t94H+B91LhFIp6jaUow0xvpHqlF3lRyNNeQ0+hmNmVZvajG78Dvwg8BRwB5rKHzQEPFRfupVLsn5Z69JoO6XYIXfeIOvRD+9DjyyPGwnPfEbiZvQr4ZHZ3K/C37r5oZjuBB4AJ4BRwp7uf7fVaGoFLCMq6HmNZQh/9hx5fXkmOwN39a+5+Q/bzU+6+mG0/4+4z7r43u+2ZvEeVav+0VK9bgREIsnc79J7y0OPLK8bCczRnYqp/WorSbTrk7HOdxyB1H0KHfmgfenx51T1NNgwtZiWSCfUQOtS4NoQeXwq0mJVEo65+/1APoeuIa5CiZKifWxMogUtQ6uz3D/UQuuq4Bu2HDvVzawJNoURoaSm7GMGp1iqGi4vp1ALUbVQ/TYm0LD25VMvFMjrpNoWiK/JEJsUr+rRTv3/9UilKjiKWqwFpCiUysV1/clAprpcem5jX4S5KLK2RSuCRSX2Eqn7/8vUrUKooGc9RiBJ4ZFIfoarfv1x5CpQqSsZzFKIEPoAQlrOtY4Ra9X6PenWfKuKNde2PvFMDs9fPsrp/lQv3XmB1/2qjkjcMdhRS578FFTFzCqV4uPFeVXWhhLLfeVURbywFrk5imRqo28b32K8Lpe5/C2ojzKmp7W2x7XeIV+cJScyxh6iqz1NnYo4o9eJhN2XudxlTHVV8TzGPYlWgLFbd/xaUwHNKvXjYTVn7XdYZl1V8T7EUuDpRgbJYdf9bUALPqantbWXtd1n97FV8T7GPYpteoCxS7f8W3L2yn3379nnMDh92n5x0N2vdHj5cd0TVKGO/zdxbY++Lf8xGf+0qvqfDTxz2yT+bdPt988k/m/TDTwz3JsO8TlHvLcWo4vsAVrxDTlURU2oRW3G0DMNcySaVq9/IYEYuYprZFjP7opk9nN3fYWZHzexEdru9yIBHEUK/dkhxhKipU1LthjldO5ZTvKUag8yB3wMca7t/AFh2973Acna/dnUuRxpiHKHSGZfDdTDU3fUgYcmVwM3s5cDbgY+0bb4dOJT9fgi4o9jQhhPKYk+hxBGyUc+4jN0wHQx1dz1IWPKOwO8Dfhe40LZtt7uvAWS3uzo90czmzWzFzFbW19dHCjaPUPq1Q4kDypvK0RTRaIbpYKi960GC0jeBm9ltwGl3f2yYN3D3g+4+7e7T4+Pjw7zEQELp1w4ljrKmcjRFNLpherLVxy0X6dSa0v4D/AHwNLAKfAs4DxwGjgN7ssfsAY73e60q2ggPH3YfG7u4NW1srPqWv6LjGLY1bnKyc7ve5ORocXR6zVFeV0S6o0sb4UB93MAtwMPZ738MHMh+PwD8Ub/nV9UHHkq/dlFxjPI/gyL7rTvFUUYft4hcrFsCH6gP3MxuAX7H3W8zs53AA8AEcAq4093P9nq++sCHM0rPdJH91t1ea9TXFZHeClnMyt0fcffbst/PuPuMu+/NbnsmbxneKAXRfv3WgxQi+71f0/q4ReqmtVAiMEpBtFe/9aCFyF7v18Q+bpG6RZvAm9TCNupZi936rXv1qnf6fLvFcfhwM/u4ReoWZQJvWgtbWWctdpsS2fg8N3++oLMnRUIS5WJWWgipGN0+xy1b4Ac/uHS7Pl+ReiR1RZ7YrhITqm5TIp2SN8R/9aEmfbfSDFEm8NiuEhOqblMzk5OdHx/z1Yea9t1KM0Q5hbL5yuPQGjmOOh+rqZmWsj7fOum7lZglNYVSdVEv9qmDQaW41Ku+W0lRlCPwsmiUli59txKzpEbgZdFVYtKl71ZSpATeJsWpA2nRdysp0hSKiEjgNIUiIpIYJXARkUgpgfegM/dEJGRb6w4gVJtPZmlf0EmFLxEJgUbgXfRaalVEJAR5rkr/UjP7NzP7kpl92cw+kG3fYWZHzexEdru9/HCrozP3RCR0eUbg3wPe5O43ADcCbzGzm2hdyHjZ3fcCy9n9ZJS1YJaISFH6JvDsosj/m929PPtx4HbgULb9EHBHKRHWRGfuiUjocs2Bm9kWM3scOA0cdfdHgd3uvgaQ3e7q8tx5M1sxs5X19fWi4i5dqmfuqbNGJB0DnYlpZlcBnwTeD3zO3a9q+9uz7t5zHlxnYtYrxWViRZqgkDMx3f07wCPAW4BnzGxP9uJ7aI3OpUBFj5bVWSOSljxdKOPZyBszuwJ4M/AV4Agwlz1sDniorCCbqIwryKizRiQteUbge4DPmNkTwL/TmgN/GPggcKuZnQBuze5LQcoYLauzRiQtfc/EdPcngNd12H4GmCkjKClntLy42HkOXJ01InHSmZiBKmO0nGpnjUhTKYEHqqw+9NnZ1iXELlxo3ZaRvGNrVYwtXpENWswqUBuJdWGhNW0yMdFK3qGPlmNbBCy2eEXa6Yo8UqjYLh4cW7zSTLoij1QitlbF2OIVaacELoWKrVUxtnhF2imBS6HKXASsjGKjFi2TmCmBS6HKalUs48zUMuMVqYKKmBIFFRulyZIvYqqXN23dioonT+r7luZKIoGXdXgt4ehVVNT3LU2VRALXMqnp61RsbKfvW5ooiQSuXt50bUyNvfvdcMUVsHNn98fq+5amSSKBq5c3TZunxs6cgeee657E9X1L0ySRwNXLW4zQCsHdpsZA37cIJJLA1cs7uhALwd2mRM6e1fctAuoDl0yIfdYhxiRSh6H7wM3sFWb2GTM7ZmZfNrN7su07zOyomZ3IbntekV7CFmIhWFNjIr3lmUJ5Afhtd38NcBPwPjN7LXAAWHb3vcBydl8iFWIhWFNjIr31TeDuvubuX8h+/x/gGHANcDtwKHvYIeCOsoKUfEYpQoY62q3iCkIisRqoiGlmU7QucPwosNvd16CV5IFdXZ4zb2YrZrayvr4+WrTS1ahFSI12ReKTu4hpZi8DPgssuvuDZvYdd7+q7e/PunvPeXAVMcujgp9IukZazMrMLgc+ASy5+4PZ5mfMbE/29z3A6aKClcGFvNhTaP3lIqnI04ViwEeBY+7+p21/OgLMZb/PAQ8VH57kFepiTyH2l4ukIs8I/Gbg3cCbzOzx7OdtwAeBW83sBHBrdj9o7SPBq69u/aQyKgx1sae8C41plC4yuK39HuDunwOsy59nig2nPBsjwY1kcubMi3/bGBVCvEW7jbgXFjrPhUM9Pd15+ss3fzcpfB8iVWjMmZjdinztUin4hVTQzBNLSPGKhCj5K/L0k2f0mcpypCH1dOeJJcSzQEVi0JgEnueMwlSWIw2ppztPLCGeBSoSg8ZMoWyeZ91sbEwnrtSl03ej70PkRY2fQtk8Ety5s/VT1QhVXRbdhXTEIBKTxozA66QRpoiMovEj8DrpossiUgYl8Aqoy0JEyqAEXgF1WYhIGZTAKxBSX3bdVMwVKY4SeAXUZdGiha1EipV8As8z4qtiVKgry9RTzNWIX1LWdzGrmOVZJEkLKVWn6mKuvltJXdJ94FpIKSxVf9b6biUV0faBj3IInGfEN+yoUIfmg6u6mKv2TUld0Al81KJXnva9YVr8VIwbTtXFXLVvSvLcvecP8Fe0rnf5VNu2HcBR4ER2u73f67g7+/bt80FMTrq3UuTFP5OT+Z5/+LD72NjFzx0ba20f5DFFxyXVGOa7FQkRsOIdcmqeEfjHgLds2nYAWHb3vcBydr9wox4C5xnxDTMq1KF5HNS+KanLVcQ0syngYXe/Lrt/HLjF3deyK9I/4u7X9nudQYuYoRahQo1LRNJUdBFzt7uvAWS3u3q88byZrZjZyvr6+kBvEuoZjKHGJSLNUnoR090Puvu0u0+Pj48P9NxQD4FDjUtEmiXoKRQRESl+CuUIMJf9Pgc8NGxgIiIynL4J3MzuB/4VuNbMnjaz9wAfBG41sxPArdl9ERGpUN+1UNz9XV3+NFNwLCIiMoCgz8QUEZHulMBFRCJV6WqEZrYOdDgFJpergW8XGE7otL9p0/6mrej9nXT3S/qwK03gozCzlU5tNKnS/qZN+5u2qvZXUygiIpFSAhcRiVRMCfxg3QFUTPubNu1v2irZ32jmwEVE5GIxjcBFRKSNEriISKSiSOBm9hYzO25mXzWzUq7+Uycze4WZfcbMjpnZl83snmz7DjM7amYnstvtdcdaFDPbYmZfNLOHs/sp7+tVZvZxM/tK9h3/bOL7+1vZv+OnzOx+M3tpavtrZn9lZqfN7Km2bV330cx+L8tfx83sl4qKI/gEbmZbgD8H3gq8FniXmb223qgK9wLw2+7+GuAm4H3ZPlZy6bqa3AMca7uf8r5+CPiUu78auIHWfie5v2Z2DfCbwHS2/PQW4C7S29+PkfNSk9l/y3cBP5U958NZXhtZ8Akc+Bngq+7+NXf/PvB3wO01x1Qod19z9y9kv/8Prf/Ar6G1n4eyhx0C7qgnwmKZ2cuBtwMfaduc6r7+GPBG4KMA7v59d/8Oie5vZitwhZltBcaA/yax/XX3fwHObtrcbR9vB/7O3b/n7l8Hvkorr40shgR+DfCNtvtPZ9uSlF0843XAowxw6brI3Af8LnChbVuq+/oqYB3462zK6CNmdiWJ7q+7fxP4E+AUsAacc/dPk+j+btJtH0vLYTEkcOuwLcneRzN7GfAJYL+7f7fueMpgZrcBp939sbpjqchW4PXAX7j764D/I/7pg66yed/bgVcCPwFcaWZ31xtV7UrLYTEk8KeBV7TdfzmtQ7KkmNnltJL3krs/mG1+JrtkHdnt6briK9DNwDvNbJXWdNibzOwwae4rtP79Pu3uj2b3P04roae6v28Gvu7u6+7+PPAg8AbS3d923faxtBwWQwL/d2Cvmb3SzF5CqxhwpOaYCmVmRmuO9Ji7/2nbn5K7dJ27/567v9zdp2h9l//s7neT4L4CuPu3gG+Y2cY1Y2eA/yDR/aU1dXKTmY1l/65naNV0Ut3fdt328Qhwl5n9iJm9EtgL/Fsh7+juwf8AbwP+E/gvYKHueErYv5+jdUj1BPB49vM2YCetavaJ7HZH3bEWvN+30LpYNinvK3AjsJJ9v/8IbE98fz8AfAV4Cvgb4EdS21/gflpz/M/TGmG/p9c+AgtZ/joOvLWoOHQqvYhIpGKYQhERkQ6UwEVEIqUELiISKSVwEZFIKYGLiERKCVxEJFJK4CIikfp/PIn58njdctEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_1 = np.linspace(0, 51, 50)\n",
    "x_2 = np.linspace(50, 101, 50)\n",
    "A = np.random.randint(10, 60, x_1.shape)\n",
    "B = np.random.randint(40, 90, x_2.shape)\n",
    "def show_point():\n",
    "    plt.scatter(x_1, A, color='blue')\n",
    "    plt.scatter(x_2, B, color='green')\n",
    "    \n",
    "show_point()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "blue_X = np.array([x_1, A])\n",
    "green_X = np.array([x_2, B])\n",
    "X = np.c_[blue_X, green_X]\n",
    "X = np.r_[X, np.ones((1, X.shape[1]))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "Y = np.ones(X.shape[1])\n",
    "Y[0:50] = 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADnCAYAAAC9roUQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de1yUdd438M/AcBZR5MyAgqOckXRQs9K0FI9Y+uy9dNuTRi2Zdte+6invnud2O2y9tO2+e2p1V2/Wcq3dZGvL8ElEzLRVt0QwNcEDCioMp0FADgJz+j1/oARyVLjmmhk+79eLF8x1/WbmO7+5+HDxm+v6XQohBIiIyDIc5C6AiGg4YegSEVkQQ5eIyIIYukREFsTQJSKyIGU/63loA1mt+fPnIzs7W+4yiHqi6G0F93TJZtXU1MhdAtEdY+gSEVkQQ5eIyIIYukREFsTQJSKyIIYuEZEFMXRJcqmpqfDz80NsbGyP64UQeP7556FWqxEfH48TJ05YuEIiy2HokuRWrVrV5/G0e/fuRVFREYqKipCeno5nn33WgtURWVZ/J0cQDdrMmTNx+fLlXtdnZmbiiSeegEKhwPTp01FfX4+KigoEBgZarkjqlcks0GY0QW80o81oht5oht5khsFkhsEoYDCbYTQJGM1mmMwCRpNo/24WMIv2n3/+DpiFgLnTzwLt/+3cWnbrthCAwK3v6LiNmz/fcqst8PPZXJ3b3t6+sy6Lb2s03m8EliYED6LnesbQJdlptVqEhIR03FapVNBqtT2Gbnp6OtLT0wEAOp3OYjXaKiEE6m8YUNPUhmvNetTe/LreYkBDiwENrQY0thrR2GpEc5sRzXoTWvRG3NCb0GIwodVggsE0fE5MVXQ6j2xulD9Dl+xTTxPpKxQ9n0WZlpaGtLQ0AIBGo5G0LlsghMC1Zj2Kdc0oqWnC1dob0Na1oKyuBZUNrahuaIPeZO7xvi5KB4x0c4KnqxKeLkq4OysRPMoZHi6OcHNyhJuzI1ydHOGqdISLkwOcHR3grGz/clE6wMmx/UvpqICTQ/t3pYMCDg7ttx0cAEcHBRwV7cscFQo4OiigUAAOCsXNr/b3WqEAHG9+V0ABhUP7ebQKheLm95vLO20Wt5b9/POt5TeXAbe17/XMXIti6JLsVCoVSktLO26XlZUhKChIxoqsk9ksUFzThBNX61FY3oBzlQ04V9mI+huGjjaODgoEjHSFarQbNGNHw9/LFf6ervDxdMEYD2d43/zycnOCq5OjjK9m+GLokuySk5OxefNmpKSk4NixY/Dy8uJ4Ltr3Ys9VNuLoxRocvViD/Ct1aGg1AgDcnR0REeCJBbGBmOA3AuG+Hgj3GYGgUa5QOvLzcWvG0CXJPfbYYzh06BBqamqgUqnwxhtvwGBo3ztbvXo1Fi5ciKysLKjVari7u2P79u0yVywfo8mMH4prsfdMBXIKq6BrbAMAhPt6YFF8IO4JHY3JoaMQ7jMCDg7W8e8y3RlFPxemHD4j6GRzNBoN8vLy5C5jSFyoakRGbim+OqlFbbMe7s6OmB3hhwcjfHGf2gdBo9zkLpHuTK9/EbmnSyQTs1kgp7ASfzpcgvwrdXByVGBedACSE4Iwa6Ivx1ztFEOXyMKMJjO+OlmOLYcu4pKuGWPHuOP/LIzCssnBGDPCRe7ySGIMXSILOlykw2+/LsSFqiZEBnhi02P3YGFcIBw5PjtsMHSJLKC09gZe312AA+eqEertji0rJmN+bIDVHDtKlsPQJZKQEAJ/O16K335dCIVCgX9fEIkn7xsHFyXHa4crhi6RRHSNbVj3xWl8e64aM8aPwbu/mIRgHoUw7DF0iSTwU9l1pH2Sh9pmPV5fEo0n7h3H42oJAEOXaMhlntTilb+fhs8IF3y5ZgZigrzkLomsCEOXaIgIIbDp24t4b/8FTB3njT8+Phk+PASMbsPQJRoCQgj8bt95bDl0CcsmB2Pjsng4KzkHAnXH0CUaJCEE3tpzFh8eKcG/TgvFW0tjOX5LvWLoEg3S2zcDd9WMcXhtSTSPvaU+8f8fokH48EgJtjFw6Q4wdInuUvaZCry1pxDzYwKwfjEDlwaGoUt0F05crcMLGSeREDIK76ckcO4EGjCGLtEdutbUhtWf5MN/pCu2PaHhFIx0R/hBGtEdMJsFXvzsFOpbDPjqyamcipHuGPd0ie7Af/+jGN9d0OE3i6MRHTRS7nLIBjF0iQYo/0ot/jPnPBbFBWLFtFC5yyEbxdAlGoAbeiNeyDiJ4FFu2LA8jkcq0F3jmC7RALyXcwFldS347Jl7MdLVSe5yyIZxT5eoH6dK6/HR0RKsmBaKqWHecpdDNo6hS9QHg8mMdV+chq+nC9YtiJS7HLIDHF4g6kP6P4pxrrIR6f9zCocVaEhwT5eoFxXXW7Dp2yIsiA3AvJgAucshO8HQJerFf+VcgNkM/O+FUXKXQnaEoUvUg4Ly6/jiRBmevG8cQrzd5S6H7AhDl+g2Qgi8vecsRrk5Yc1stdzlkJ1h6BLd5uD5avzz0jW88NAEeLnxwzMaWgxdok5MZoENWecQ5uOBf502Vu5yyA4xdIk6yfqpAkXVTXhp3kReWJIkwa2KJJednY2IiAio1Wps3Lix2/rr169jyZIlmDRpEmJiYrB9+3YZqmyftnHztxeh9huBBbGBstRA9o+hS5IymUxYu3Yt9u7di8LCQuzcuROFhYVd2vzhD39AdHQ0Tp06hUOHDuGll16CXq+3eK05hZU4X9WI52areSUIkgxDlySVm5sLtVqN8PBwODs7IyUlBZmZmV3aKBQKNDY2QgiBpqYmeHt7Q6m07MmSQghs+vYixo1xx+J47uWSdBi6JCmtVouQkJCO2yqVClqttkub5557DmfPnkVQUBDi4uLwwQcfwMGh500zPT0dGo0GGo0GOp1uyOr89lw1CsobsGa2GkpH/lqQdLh1kaSEEN2W3T4X7b59+5CQkIDy8nKcPHkSzz33HBoaGnp8vLS0NOTl5SEvLw++vr5DVuPvv70I1Wg3PHpP8JA8JlFvGLokKZVKhdLS0o7bZWVlCAoK6tJm+/btWLZsGRQKBdRqNcLCwnDu3DmL1Xj8ch1OldbjmVnj4cS9XJIYtzCSVGJiIoqKilBSUgK9Xo+MjAwkJyd3aRMaGooDBw4AAKqqqnD+/HmEh4dbrMaPjpTAy80J/2OyymLPScMXp3YkSSmVSmzevBlJSUkwmUxITU1FTEwMtm7dCgBYvXo11q9fj1WrViEuLg5CCLzzzjvw8fGxSH2ltTeQU1iJtJnj4ebMS6mT9BQ9jbl10udKIjlpNBrk5eUN6jHe3lOIj45exuFXZiNolNsQVUaEXo855PACDVvNbUZkHC/F/NgABi5ZDEOXhq0vTpShsdWI1PvC5C6FhhGGLg1LZrPAn49exiSVFyaHjpK7HBpGGLo0LH1ffA3FNc1Ydd+4bscNE0mJoUvD0s7cq/Byc+LENmRxDF0admqb9cgpqMKj9wTD1YmHiZFlMXRp2PnyRBn0JjMemxoqdyk0DDF0aVgRQiDjeCnuCR2FiABPucuhYYihS8NK/pU6XKxuwmOJ3MsleTB0aVjZmVsKD2dHLOKcuSQThi4NGw2tBuz5qRzJCcHwcOG0IyQPhi4NG3t/qkCrwYxfJob035hIIgxdGjZ2/ahFuI8HJqm85C6FhjGGLg0L2voW/FBci0fuCeYZaCQrhi4NC5kn26/L9kgCL8dD8mLokt0TQmDXCS00Y0cjdIy73OXQMMfQJbtXUN6AouomPMKLTpIVYOiS3dv1oxZOjgos5rG5ZAUYumTXjCYzdp8qx+wIP4xyd5a7HCKGLtm374uvQdfYxqEFshoMXbJrX5+qgIezI+ZE+sldChEAhi7ZMYPJjOyCSsyN9ue8uWQ1GLpkt45erMH1FgMWxQfJXQpRB4Yu2a09pyvg6aLEzIk+cpdC1IGhS3ZJbzRjX0El5sb4w0XJoQWyHgxdsktHLurQ0GrksblkdRi6ZJe+Pl2Bka5K3K/2lbsUoi4YumR3Wg0m7C+oQlJMAJyV3MTJunCLJLtzuKgGjW1GXpKHrBJDl+xO9plKjHRVYsZ4HrVA1oehS5LLzs5GREQE1Go1Nm7c2GObQ4cOISEhATExMZg1a9ZdP5fBZMY3Z6vwcJQ/hxbIKvHqfCQpk8mEtWvXYv/+/VCpVEhMTERycjKio6M72tTX12PNmjXIzs5GaGgoqqur7/r5jhXX4nqLAfNjA4aifKIhx10BklRubi7UajXCw8Ph7OyMlJQUZGZmdmnz6aefYtmyZQgNDQUA+Pnd/TwJ2QUVcHNyxMyJPGqBrBNDlySl1WoREvLz1XdVKhW0Wm2XNhcuXEBdXR0efPBBTJkyBR9//HGvj5eeng6NRgONRgOdTtdlndkssK+gCrMjfTnXAlktDi+QpIQQ3ZbdfmFIo9GI/Px8HDhwAC0tLbj33nsxffp0TJw4sdt909LSkJaWBgDQaDRd1p24WgddYxuSYji0QNaLoUuSUqlUKC0t7bhdVlaGoKCgbm18fHzg4eEBDw8PzJw5E6dOneoxdPuSfaYSzo4OnMaRrBqHF0hSiYmJKCoqQklJCfR6PTIyMpCcnNylzdKlS3H48GEYjUbcuHEDx44dQ1RU1B09jxAC2QWVuH+CDzxdnYbyJRANKe7pkqSUSiU2b96MpKQkmEwmpKamIiYmBlu3bgUArF69GlFRUZg/fz7i4+Ph4OCAp59+GrGxsXf0PAXlDSira8HzcyZI8TKIhoyipzG3TvpcSSQnjUaDvLw8AMB7Oeex+eBF5P3HXHh78FpoJDtFbys4vEB2IaewConjvBm4ZPUYumTzrl67gXOVjZgb7S93KUT9YuiSzcsprAQAzIvmoWJk/Ri6ZPNyCqsQGeCJ0DHucpdC1C+GLtm02mY98i7XYh6HFshGMHTJph04WwWzAObxLDSyEQxdsmk5hVUI8nJFTNBIuUshGhCGLtksswAOF+kwN9q/23wORNaKZ6SRzWpqNUBvMHNogWwK93TJZjW0GuHpqsTUMG+5SyEaMIYu2SSTWaCh1YA5kX5wcuRmTLaDWyvZpBNX62AyC56FRjaHoUs2aX9hFRQAZvGyPGRjGLpkc4QQ2F9YBQ8XJefOJZvD0CWbc0nXhJKaZox0Y+CS7WHoks3JKawCAIx05RGPZHsYumRz9hdWIS7Yi0ctkE3iVks2pbqxFSdL63nUAtkshi7ZlANnqyEEGLpksxi6ZFP2F1ZBNdoNkQGecpdCdFcYumQzmtuMOHKxBvOiAzjBDdkshi7ZjH9c0EFvNGNeDIcWyHYxdMlm5BRWYbS7EzRjR8tdCtFdY+iSTTCYzDhwtgpzIv2h5KFiZMO49ZJNOF5Si4ZWI4cWyOYxdMkm5BRWwdXJATMncIIbsm0MXbJ6QgjkFFTigQm+cHN2lLscokFh6JLVKyhvQPn1Vp4QQXaBoUtWL6ewCg4K4KFIP7lLIRo0hi5ZvX1nKqEZ540xI1zkLoVo0Bi6ZNWKdU04X9WIBbG84i/ZB4YuSS47OxsRERFQq9XYuHFjr+2OHz8OR0dH/P3vf+9Ytq+gfe7cJF5mnewEQ5ckZTKZsHbtWuzduxeFhYXYuXMnCgsLe2y3bt06JCUldVmeXVCJSSovBI1ys1TJRJJi6JKkcnNzoVarER4eDmdnZ6SkpCAzM7Nbu02bNmH58uXw8/v5w7Ly+hacKq1HEocWyI4wdElSWq0WISEhHbdVKhW0Wm23Nrt27cLq1au7LM8pqAQAzO80tJCeng6NRgONRgOdTidh5UTSYOiSpIQQ3ZbdPi3jr3/9a7zzzjtwdOx64sPeM5WY6D8C4b4jOpalpaUhLy8PeXl58PXl2Wlke3hlP5KUSqVCaWlpx+2ysjIEBQV1aZOXl4eUlBQAQE1NDbKystAilDh+2QnPzVZbtF4iqTF0SVKJiYkoKipCSUkJgoODkZGRgU8//bRLm5KSko6fV61ahcWLF8MYOhnm/J84nkt2h6FLklIqldi8eTOSkpJgMpmQmpqKmJgYbN26FQC6jePesvdMJUK83RAdONKS5RJJTtHTmFsnfa4kkkJdsx6Jb3+Dpx4Iw6sLonptp9FokJeXZ8HKiAas1+tJ8YM0sjr7CiphNAssiQ/qvzGRjWHoktXZ81MFxo5xR0wQhxbI/jB0yapca2rDPy9dw+L4QF7xl+wSQ5esSnZBJUxmgUVxHFog+8TQJauy53QFwn09EBXoKXcpRJJg6JLV0DW24Yfia1gcx6EFsl8MXbIa2WcqYBbAIh61QHaMoUtW4/+droDabwQiAji0QPaLoUtWoazuBnJLavFIAvdyyb4xdMkqZJ4sBwAsTQiWuRIiaTF0SXZCCHx5ogxTx3kjxNtd7nKIJMXQJdmd0Tbgkq4Zj9zDvVyyfwxdkt2uH7VwdnTAorhAuUshkhxDl2RlNJmx+1Q55kT6wcvdSe5yiCTH0CVZHblYg5qmNg4t0LDB0CVZ7fpRCy83J8yO5PXOaHhg6JJsrt8wIPtMJRbHB8JF6dj/HYjsAEOXZJN5Sos2oxkpiaFyl0JkMQxdkoUQAjtzSxETNBJxKi+5yyGyGIYuyeJ02XWcrWhAylTu5dLwwtAlWWQcL4WbkyOWcq4FGmYYumRxzW1G7D6pxaL4QIx05bG5NLwwdMni9pyuQLPehMemhshdCpHFMXTJ4j7NvQq13whMDh0tdylEFsfQJYs6VVqPk6X1WDEtlJfkoWGJoUsWtf1oCTxdlPiFhkMLNDwxdMliqhpa8fXpCvxCE4IRLkq5yyGSBUOXLOYvP1yBSQismjFO7lKIZMPQJYtoNZjw12NX8XCUP0LH8OoQNHwxdMkidp8sR22zHqn3hcldCpGsGLokub1792LdR9lAvRaHPvtTt/V//etfER8fj/j4eMyYMQOnTp2SoUoiy2DokqRMJhPWvr0FwisIG5+ch4yMnSgsLOzSJiwsDN999x1Onz6N9evXIy0tTaZqiaTH0CVJHTt2DM73LEWItxuWa0KRkpKCzMzMLm1mzJiB0aPbT5SYPn06ysrK5CiVyCIYuiSpg2cr0eoRgDUPquHk6ACVSgWtVttr+w8//BALFizodX16ejo0Gg00Gg10Op0UJRNJigdLkmSEENhfroSL6QaWT1Z1LO/tTLSDBw/iww8/xJEjR3p9zLS0tI7hB41GM7QFE1kA93RJMt8XX8PlZkd4lR+Ds7J9UysrK0NQUPfpHE+fPo2nn34amZmZGDNmjKVLJbIYhi5J5vcHiuDn6QLdsd0oKSmBXq9HRkYGkpOTu7S7evUqli1bhk8++QQTJ06UqVoiy+DwAkniHxd0+KG4Fr9ZHI2Ae95HUlISTCYTUlNTERMTg61btwIAVq9ejTfffBPXrl3DmjVrAABKpRJ5eXlylk8kGYUQoq/1fa4k6onJLLDo94fRrDfimxdnSXalX41Gw3Ama9XrFHocXqAh90V+Gc5VNmLd/EheWp3oNgxdGlLNbUb8Z8553BM6CoviAuUuh8jqMHRpSP3pcDGqG9vwH4uiOEk5UQ8YujRkyutb8N/fFWNRXCCmjPWWuxwiq8TQpSEhhMD6r84AAP59QaTM1RBZL4YuDYk9P1XgwLlqvDRvIkK8OV8uUW8YujRo9Tf0eH13AeJVXrwqBFE/eHIEDdrbe86i7oYBH6dOg9KRf8eJ+sLfEBqUA2er8Hl+GZ6ZGY7ooJFyl0Nk9Ri6dNfK61vw0uenEB04Es8/NEHucohsAkOX7orRZMbzO3+EwWjGH1ZMhqsTzzwjGgiO6dJdeW//BeRdqcMHKQkI8/GQuxwim8E9Xbpj+wur8MdDl5CSGIKlCcFyl0NkUxi6dEdOldbj33aewCSVF15bEiN3OUQ2h6FLA1ZaewNP7TgOX08XbFuZCDdnjuMS3SmO6dKA1DXrsXJ7Lgwmgb89ORW+ni5yl0Rkkxi61K9rTW1Yse0Yyupa8JenpmG87wi5SyKyWQxd6lN1QytWbDuG0rob+HClBlPDOHsY0WAMakz3888/R0xMDBwcHLpdNmXDhg1Qq9WIiIjAvn37erx/bW0t5s6diwkTJmDu3Lmoq6sbTDm9+uUvf4mEhAQkJCRg3LhxSEhI6LHduHHjEBcXh4SEBItc3vv1119HcHBwR21ZWVk9tsvOzkZERATUajU2btwoeV0vv/wyIiMjETv1ATzw+hcoq7uB7aum4oEJvl3aWaK/+nvtQgg8//zzUKvViI+Px4kTJySp43alpaWYPXs2oqKiEBMTgw8++KBbm0OHDsHLy6vj/X3zzTctUlt/74scfXb+/PmOfkhISMDIkSPx/vvvd2ljqf5KTU2Fn58fYmNjO5YNNIuG5HdRCNHXV58KCwvFuXPnxKxZs8Tx48c7lhcUFIj4+HjR2toqiouLRXh4uDAajd3u//LLL4sNGzYIIYTYsGGDeOWVV/p7ykF78cUXxRtvvNHjurFjxwqdTid5Dbe89tpr4t133+2zjdFoFOHh4eLSpUuira1NxMfHi4KCAknr2rdvn/jhYrWY8tscoX7lK/HkK2/12E7q/urvtU+ZMkXs2bNHzJ8/X5jNZvH999+LqVOnSlZPZ+Xl5SI/P18IIURDQ4OYMGFCt/fl4MGDYtGiRRapp7P+3he5+uwWo9Eo/P39xeXLl7sst1R/fffddyI/P1/ExMR0LBtIFt3h72KvuTqoPd2oqChERER0W56ZmYmUlBS4uLggLCwMarUaubm5PbZbuXIlAGDlypX46quvBlNOv4QQ+Oyzz/DYY49J+jxDKTc3F2q1GuHh4XB2dkZKSgoyMzMlfc660ZF4/KPjGOGixEuTzGgrK5T0+XozkNeemZmJJ554AgqFAtOnT0d9fT0qKiokry0wMBCTJ08GAHh6eiIqKgparVby5x0KcvXZLQcOHMD48eMxduxYiz1nZzNnzoS3d9dhsoFk0VD9LkpyyJhWq0VISEjHbZVK1eMGWVVVhcDA9utoBQYGorq6WopyOhw+fBj+/v6YMKHneQIUCgXmzZuHKVOmID09XdJabtm8eTPi4+ORmpra4780A+3LodDUZsS6v5/Gui9+wvTwMchcez/27PwICxYs6LG91P01kNduyf7pzeXLl/Hjjz9i2rRp3dZ9//33mDRpEhYsWICCggKL1NPf+yJ3n2VkZPS64yNHfwEDy6Kh6rd+P0h7+OGHUVlZ2W3522+/jaVLl/Z4H9HDZd2lvl7WQOrcuXNnn3u5R48eRVBQEKqrqzF37lxERkZi5syZktX17LPPYv369VAoFFi/fj1eeuklfPTRR13aSdWXt9dl8g5Da8K/AO6jseZBNV6cOxHvbNwApVKJFStW9PgYUvRXZwN57XJsa501NTVh+fLleP/99zFyZNdZ1iZPnowrV65gxIgRyMrKwiOPPIKioiLJa+rvfZGzz/R6PXbv3o0NGzZ0WydXfw3UUPVbv6H7zTff3PGDqlQqlJaWdtwuKytDUFBQt3b+/v6oqKhAYGAgKioq4Ofnd8fPNdA6jUYjvvzyS+Tn5/fa5laNfn5+ePTRR5GbmzvoEBlo//3qV7/C4sWLuy0faF/ebV21zXr83/0X8JdjVxDq7Y73/mUSpoz1xo4dO/D111/jwIEDvW5YUvRXZwN57VL1z0AYDAYsX74cK1aswLJly7qt7xzCCxcuxJo1a1BTUwMfHx9J6+rvfZGzz/bu3YvJkyfD39+/2zq5+gsYWBYNVb9JMryQnJyMjIwMtLW1oaSkBEVFRZg6dWqP7Xbs2AEA2LFjR697zkPhm2++QWRkJFQqVY/rm5ub0djY2PFzTk5Ol083pdB5HG3Xrl09Pl9iYiKKiopQUlICvV6PjIwMJCcnD/q59UYzPjxSggffPYhPc69i5b3jkPX8A5gy1hvZ2dl45513sHv3bri793zpHUv010Bee3JyMj7++GMIIfDDDz/Ay8ur499EKQkh8NRTTyEqKgovvvhij20qKys79o5yc3NhNpsxZswYSesayPsiV58Bff+3KUd/3TKQLBqy38W+PmXr71PAL7/8UgQHBwtnZ2fh5+cn5s2b17HurbfeEuHh4WLixIkiKyurY/lTTz3VcaRDTU2NmDNnjlCr1WLOnDni2rVr/T3lXVu5cqXYsmVLl2VarVYsWLBACCHEpUuXRHx8vIiPjxfR0dHirbd6/sR+KD3++OMiNjZWxMXFiSVLlojy8vJudQnR/mnzhAkTRHh4+KDratEbxY5/logZGw6Iseu+Fo9v+0Gcr2zo0mb8+PFCpVKJSZMmiUmTJolnnnmmW12W6q+eXvuWLVvEli1bxJQpU4TZbBZr1qwR4eHhIjY2tstRNFI6fPiwACDi4uI6+mnPnj0dtQkhxKZNm0R0dLSIj48X06ZNE0ePHpW8rt7el851ydVnzc3NwtvbW9TX13csk6O/UlJSREBAgFAqlSI4OFhs27at1ywaxO9ir7mqED2MU3TO5DuPcbJGlddb8VleKT7+/jJqmvSYHDoK//bQBDw40deiY6BDSaPRdDs+nMhK9PpLxTPS7FiL3oTvLlTj87wyHDxfDbMAZk70xZoHx2NamLfNhi2RLWPo2hldYxuOXqxBTmElDp7TocVggq+nC1bPGo9fJoZg7BhOOE4kJ4aujau43oITV+px4modjl6swbnK9g9RfEa4YPmUYCyIDcS0MG9epZfISjB0bURjqwGXa26guKYJ5yobca6iAYUVDahqaAMAuCgdMGXsaLycFIEHJvggJsgLjg4cPiCyNpLu/ljqrK47ZU11CSFwvcWAkppmrP/9n5H1UwW2Hy3Bhr1n8dynJ/DoH48i8e1vEPd6DpZsPoIXMk5i2+FiVFxvxYzxPnhtSTQy196Hn15Pwqe/mo61s9WIV40a0sAdbH9JNWmNTqcbVF1SsabtqzPWdWekqkvSoxes9dPl3uoSQsAsAKPZDLMZMAkBk0nAaDbDZBYwmAUMRjOMZjP0RgGDyQy9yQy9sf2rzWhCm9GMFr0JrQYTWgxm3NAb0dxmwg29EY1tRjS1GtHYakB9iwENLQZcbzHAYOrezU6OCgSNckPwza8wXw+E+3ggzCd60GAAAAKGSURBVGcEwnw84Ky03HDBYN5Hk8mEiRMnYv/+/VCpVEhMTMTOnTsRHR3d0SYrKwubNm1CVlYWjh07hhdeeAHHjh3r97E9PDzQ3Nx8V3VJyda2e7nZaV2WPXqhWNeEtE/yUTd9LR5+77seT58Dbkt00f3HW/cTAG49hID4+efO9xGio92tNmZx8x4CMN8MVLMQaHrwVUT/JvvnZWYBkxDo++/P3VEoAA9nJdycHeHpqoSnixKerk4I9HLDSDcnjHJ3whgPZ3h7OOP1V/8X/vbndPh5umC0uzMc7GB4oPMkIQA6JgnpHLq9TcBiqQP2iSypzz3d+fPni5qamjt+UIPJjIrrrWhra4OLy91d1qXHuOlhoaKXPyi3Hw3V+WZrSyvc3N1+Xqb4eb3i5g1Fp+UKRfuzKG49rkIBh5s/d6xTKOCg+Pm7g0IBB4WiWx190el08PX17b+hhQ2mrrq6OjQ0NHTMKHXt2jU0NzcjNDS0o83FixcREBCAESPar0hx4cIFqFSqHs+G0+l0uLVNtrW19To3spzs8X2Ukj3WlZ+fv08IMb+ndTw5giT1+eefY9++fdi2bRsA4JNPPkFubi42bdrU0WbRokV49dVXcf/99wMAHnroIfzud7/DlClT+nxsa/23lAh9DC/wOCKSlLVPWkNkaQxdkpQ1T1pDJAcep0uSUiqV2Lx5M5KSkmAymZCamoqYmBhs3boVALB69WosXLgQWVlZUKvVcHd3x/bt22Wumkg6HNMlm8UxXbJiHNMlIrIGDF0iIgti6BIRWRBDl4jIgvr7II3IaikUiuzezvohslYMXSIiC+LwAhGRBTF0iYgsiKFLRGRBDF0iIgti6BIRWdD/B8UNhILHWpYhAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "def f(_X, _W):\n",
    "    return np.dot(_W, _X)\n",
    "\n",
    "def g(_Z):\n",
    "    return 1 / (1 + np.exp(- _Z))\n",
    "\n",
    "def h(_X, _W):\n",
    "    f_x = f(_X, _W)\n",
    "    return g(f_x)\n",
    "\n",
    "def G(_W, _X, _Y):\n",
    "    h_x = h(_X, _W)\n",
    "    return (h_x - _Y) * _X\n",
    "\n",
    "def update_W(_W, _G, learn_rate=np.array([0.001, 0.002, 0.003])):\n",
    "    return _W - (learn_rate * _G)\n",
    "\n",
    "\n",
    "def cost(_W, _X, _Y):\n",
    "    h_x = h(_X, _W)\n",
    "    return _Y * np.log(h_x) + (1 - _Y) * np.log(1 - h_x)\n",
    "    \n",
    "_x = np.linspace(-10, 10, 100)\n",
    "_y = g(_x)\n",
    "\n",
    "def sigmod():\n",
    "    fig, ax = plt.subplots()\n",
    "    ax.spines['right'].set_color('none')\n",
    "    ax.spines['bottom'].set_color('none')\n",
    "    ax.xaxis.set_ticks_position('top')\n",
    "    ax.yaxis.set_ticks_position('left')\n",
    "    ax.spines['left'].set_position(('data',0))\n",
    "    ax.spines['top'].set_position(('data',0))\n",
    "    plt.plot(_x, _y)\n",
    "sigmod()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(_X, _Y, batch, times):\n",
    "    guess_W = np.array([1.0, 2.0, 3.0])\n",
    "    _, length = _X.shape\n",
    "\n",
    "    loss = []\n",
    "    _g = []\n",
    "    _back = []\n",
    "    for iter_index in range(times):\n",
    "        data_index = np.random.choice(length, batch)\n",
    "        x_data = _X[:, data_index]\n",
    "        y_data = _Y[data_index]\n",
    "        _G = G(guess_W, _X, _Y).mean()\n",
    "        _g.append(_G)\n",
    "        guess_W = update_W(guess_W, _G)\n",
    "        loss.append(cost(guess_W, _X, _Y).mean())\n",
    "        display.clear_output(wait=True)\n",
    "        plt.subplot(121)\n",
    "        plt.plot(np.arange(len(loss)), loss, color='red', label='loss')\n",
    "        plt.legend()\n",
    "        plt.subplot(122)\n",
    "        plt.plot(np.arange(len(_g)), _g, color='red', label='grad')\n",
    "        plt.legend()\n",
    "        _back.append(h(x_data, guess_W).mean())\n",
    "        plt.show()\n",
    "    return guess_W\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD4CAYAAAD4k815AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAcP0lEQVR4nO3dfZRU9Z3n8feXprERkIeGCKEhDUYnyRhBaNwZH4iTMMZIBB8OQc7GJWpCZmZ1x7g5GbNklBydJANjxpyzHiLjuMEsJpkTcSeRuKNm13BMNGODYHjQ8Tlp6NHuApSHbZuG7/5RVbHFbrrpur+69/768zqnT1VX3b73W11Vn/72r+79XXN3REQkHkPSLkBERJKlYBcRiYyCXUQkMgp2EZHIKNhFRCIzNI2Njh8/3hsbG9PYtAwCmzZtanf3CWlsW69tCam/r+1Ugr2xsZHm5uY0Ni2DgJm9lta29dqWkPr72tZQjIhIZBTsIiKRUbCLiEQmlTF2EZGBOHz4MC0tLXR0dKRdSlB1dXU0NDRQW1s7oJ9XsMugZ2b3Ap8G3nD3M0u3jQN+BDQCrwKfcfe9adUoRS0tLYwaNYrGxkbMLO1ygnB3CoUCLS0tTJs2bUDr0FCMCHwPuPiY224Gfu7upwM/L30vKevo6KC+vj7aUAcwM+rr6yv6r0TBLoOeu28E9hxz80Jgben6WuCyqhYlvYo51MsqfYzxDcX86lfw6KPvfH/stMSapjgOQ4fCX/91yC2c6u6tAO7eambv621BM1sGLAOYOnXqexd46SVYu7b7Dxz/el/3J/Fz1djGySdDfT00NsKZZxafM6mKuH7Tv/oVfPzj8PbbaVciodXVhQ72fnP3NcAagKampvd2Dq+8ArffXl64mqVlx8iR8MUvwm23wfDhaVeTGeUD2saPH5/oeuMJdnf4zGdgyhR48slip1A2CP51k8S9bmaTSt36JOCNAa9p3jw4evT4y5QDv3vw93X9RJat9s+5w8GD0N4OL7wAP/sZ3HEHvPgiPPhg1O/Jrq4uhqb830k8wf6738GuXbB6NST8108GpZ8AS4FvlS7/OejWehrqyLvx4+EDH4DZs+Gqq+Dss+Gmm+Chh+DSS9OubsBuu+021q1bx5QpUxg/fjyzZ8/moYce4txzz+WXv/wlCxYs4IwzzuD222+ns7OT+vp61q1bx6mnnkqhUGDJkiW0tbVxzjnnEOoMdvEE+/btxcs//MN065DcMbMfABcC482sBbiVYqD/k5ldB/wWWJRehZG4/nr4xjfg+99PJthvvBG2bKl8Pd3NnAl33tnr3c3NzTzwwAM888wzdHV1MWvWLGbPng3Avn37+MUvfgHA3r17eeqppzAz7rnnHlauXMkdd9zB17/+dc4//3xuueUWNmzYwJo1a5Ktv0TBLoOeuy/p5a5PVLWQ2NXWwqJF8L3vwZEjUFOTdkUn7IknnmDhwoUML31OcGm3P1CLFy/+/fWWlhYWL15Ma2srnZ2dv98ffePGjaxfvx6A+fPnM3bs2CB1xhPs27bBxIkwblzalYhIb+bMKQ6XvvIKfPCDla3rOJ11KMcbOhkxYsTvr99www3cdNNNLFiwgMcff5wVK1b8/r5q7K4Zz37s27erWxfJug9/uHj53HPp1jFA559/Pj/96U/p6OjgwIEDbNiwocfl3nzzTSZPngzA2m67us6dO5d169YB8PDDD7N3b5iDmeMI9qNHYccOBbtI1n3oQ8XLnTvTrWOA5syZw4IFC5gxYwZXXHEFTU1NjB49+j3LrVixgkWLFnHBBRe8a1fGW2+9lY0bNzJr1iweeeSRno97SIK7V/1r9uzZnqhXXnEH97vvTna9kktAs6fwuvYQr+0YTZzofs01A/rRHTt2JFzMidu/f7+7ux88eNBnz57tmzZtCrKdnh5rf1/bcYyxv/pq8fK001ItQ0T6oaEBXn897SoGbNmyZezYsYOOjg6WLl3KrFmz0i7pPSoKdjNbBKwAPgyc4+7pnBNs9+7i5fvfn8rmReQEjBkD+/alXcWA3X///WmX0KdKx9i3AVcAGxOoZeBaW4uXkyalWoaI9MPo0RUFuw+CaRkqfYwVBbu773T35yuqIAmtrcX5J3r4EENEMqaCjr2uro5CoRB1uHtpPva6uroBr6NqY+x9zoBXid27i916TIdji8SqgmBvaGigpaWFtra2hIvKlvIZlAaqz2A3s8eAiT3ctdzd+z1/hvc1A14lWls1DCOSF2PGwKFD0NkJw4ad0I/W1tYO+KxCg0mfwe7u86pRSEV274YZM9KuQkT6Y8yY4uWbb8KECenWEqk4DlBSxy6SH+Vgz/GeMVlXUbCb2eWl2fD+GNhgZv+STFkn4MAB2L9fwS6SFwr24Cr68NTdHwQeTKiWgSnv6qh92EXyoftQjASR/6EY7cMuki/l3ZLVsQejYBeR6ho5sni5f3+6dUQsnmCf2NMemSKSOeVdHA8fTreOiOU/2Nvaimdi0Qk2RPJBwR5c/oO9vR3q62FI/h+KyKBQW1u87OxMt46I5T8N29p0kINInpQ7dgV7MHEEe7czlIhIxmkoJrg4gl0du0h+1NQUL9WxB5P/YG9vV7CL5IlZsWtXsAeT72A/cgT27FGwi+TNsGEaigko38FeKIC7gl0kb2pr1bEHlO9gL0+2rw9PRfJFQzFBxRHs6thF8kVDMUHlO9jb24uXCnaRfNFQTFD5DnZ17CL5pKGYoOII9vr6dOsQkROjoZig8h/so0ef8AlxRSRlGooJKt/BroOTJDAz+5KZbTezbWb2AzOrS7umKGgoJqh8B7umE5CAzGwy8F+AJnc/E6gBrkq3qkhoKCYoBbvI8Q0FhpvZUOBkYHfK9cRBQzFB5T/YdXCSBOLuu4C/A34LtAJvuvsjxy5nZsvMrNnMmtvKH+jL8WkoJqj8Bru7xtglKDMbCywEpgHvB0aY2WePXc7d17h7k7s3TdDrsX80FBNUfoP9rbeKLwy9kSScecAr7t7m7oeB9cC5KdcUBw3FBJXfYNfBSRLeb4E/MrOTzcyATwA7U64pDurYg8p/sGuMXQJx918DPwY2A7+h+H5Zk2pRsdAYe1BDK/lhM1sFXAp0Ai8B17j7viQK65M6dqkCd78VuDXtOqKjoZigKu3YHwXOdPezgH8Dvlp5Sf2kCcBE8ktDMUFVFOzu/oi7d5W+fQpoqLykflLHLpJfGooJKskx9muBh3u7M/F9fdvaYPhwGDGi8nWJSHVpKCaoPoPdzB4rzZNx7NfCbsssB7qAdb2tJ/F9fXVwkkh+lYdi3NOuJEp9fnjq7vOOd7+ZLQU+DXzCvYrPkg5OEsmvYcOKoX7kCAytaB8O6UGle8VcDPwV8DF3P5RMSf2keWJE8qu2tnjZ2algD6DSMfb/DowCHjWzLWb23QRq6h8NxYjkV/kcChpnD6KiP5Xu/sGkCjlhhYKCXSSvamqKl0ePpltHpPJ55GlnJ+zfr1PiieTVkFL0HDmSbh2RymewFwrFS3XsIvmkjj2ofAe7OnaRfFLHHpSCXUSqrxzs6tiDULCLSPVpKCaofAZ7eQIwjbGL5JOGYoLKZ7CrYxfJN3XsQeU32IcPL36JSP6oYw8qv8GuYRiR/FLHHlQ+g729XcMwInmmjj2ofAZ7oaBgF8kzdexBKdhFpPrUsQeVz2Bvb9cYu0ieqWMPKn/BfuQI7N2rjl0kz9SxB5W/YN+3r3jmFQW7SH6pYw8qf8Gug5NE8k8de1D5C3ZNJyCSf+rYg8pfsKtjF8k/dexBKdhFpPrUsQelYBeR6lPHHlT+gr29HYYOhVNOSbsSERkodexB5S/Yy0edmqVdiYgMlDr2oPIb7CKSX+rYg1KwixyHmY0xsx+b2XNmttPM/jjtmqKgjj2o/AW75omR6voO8L/d/UPADGBnyvXEQR17UEPTLuCEqWOXKjGzU4C5wOcA3L0T6EyzpmiUO3YFexAVdexmdpuZPWtmW8zsETN7f1KF9chdwS7VNB1oA/6HmT1jZveY2YhjFzKzZWbWbGbNbW1t1a8yj8odu4Zigqh0KGaVu5/l7jOBh4BbEqipd/v3w+HDGoqRahkKzAJWu/vZwEHg5mMXcvc17t7k7k0TJkyodo35pI49qIqC3d3f6vbtCMArK6cPOjhJqqsFaHH3X5e+/zHFoJdKqWMPquIxdjP7G+A/AW8Cf3Kc5ZYBywCmTp06sI0p2KWK3P3fzex3ZvYH7v488AlgR9p1RUEde1B9duxm9piZbevhayGAuy939ynAOuD63taTyL+rCnapvhuAdWb2LDAT+EbK9cRBHXtQfXbs7j6vn+u6H9gA3FpRRcejKXulytx9C9CUdh3RUcceVKV7xZze7dsFwHOVldMHdewicdABSkFVOsb+LTP7A+Ao8BrwZ5WXdByFQnGOmLFjg25GRALTAUpBVRTs7n5lUoX0S6EAY8a886IQkXxSxx5UvqYU0HQCInFQxx5UvoJdR52KxEEde1AKdhGpPnXsQSnYRaT61LEHla9g1xi7SBzUsQeVn2Dv6IBDh9Sxi8RAHXtQ+Ql2HZwkEg917EHlL9g1FCOSf+rYg8pPsJfniVHHLpJ/6tiDyk+wayhGJB5mxUt17EEo2EUkHTU16tgDyU+wayhGJC5DhqhjDyQ/wV4owMiRcNJJaVciIklQxx5MvoJd3bpIPIYMUbAHomAXkXTU1GgoJpD8BLumExCJizr2YPIT7OrYReKijj0YBbuIpEMdezD5CPauLti3T8EuEhN17MHkI9j37CleaoxdJB7q2IPJR7DrqFOR+KhjD0bBLiLpUMceTL6CXUMxIvFQxx5MPoJd88SIxEcdezD5CHYNxYjERx17MPkJ9mHDYMSItCsRkaSoYw8mkWA3sy+bmZtZmEHw8nQC5cn5RST/1LEHU3Gwm9kU4E+B31ZeTi901KmkyMxqzOwZM3so7Vqioo49mCQ69r8HvgJ4AuvqmYJd0vWXwM60i4iOTrQRTEXBbmYLgF3uvrUfyy4zs2Yza25razuxDSnYJSVm1gDMB+5Ju5bo6EQbwQztawEzewyY2MNdy4H/BlzUnw25+xpgDUBTU9OJdfeaslfScyfF/0hH9baAmS0DlgFMnTq1SmVFQB17MH0Gu7vP6+l2M/soMA3YasUPNRuAzWZ2jrv/e2IVuhfnilHHLlVmZp8G3nD3TWZ2YW/LVdS0DGbq2IPpM9h74+6/Ad5X/t7MXgWa3L09gbre8eabxb/qCnapvvOABWZ2CVAHnGJm/9PdP5tyXXFQxx5M9vdj18FJkhJ3/6q7N7h7I3AV8H8U6glSxx7MgDv2Y5Ve/MkrTyegMXaRuKhjDyaxYA9GHbtkgLs/DjyechlxqakpnkRHEqehGBFJhw5QCiY/wa6hGJG4aEqBYLIf7O3txb/so0enXYmIJEkdezDZD/ZCAcaNK74IRCQe6tiDyX5aajoBkTipYw8mH8Gu8XWR+KhjDyb7wd7ero5dJEbq2IPJfrBrKEYkTurYg8l2sLsr2EVipY49mGwH+6FD0NGhMXaRGKljDybbwa6jTkXipY49GAW7iKRDHXswCnYRSYc69mCyHeyaslckXurYg8l2sKtjF4mXOvZg8hHs48alW4eIJE8dezDZD/bRo6G2Nu1KRCRp6tiDyXawazoBkXipYw8m28Guo05F4qWOPRgFu4ikQx17MNkPdu3qKBIndezBZDvYNcYuEi8FezDZDfbOTti/X8EuEisNxQST3WDfs6d4qWAXiZM69mAqCnYzW2Fmu8xsS+nrkqQK03QCIpGrqSkGu3valURnaALr+Ht3/7sE1vNumk5AJG5DSn2lO5ilW0tksjsUo2AXiVtNTfFS4+yJSyLYrzezZ83sXjMb29tCZrbMzJrNrLmtra3vtSrYReJW7tg1zp64PoPdzB4zs209fC0EVgOnATOBVuCO3tbj7mvcvcndmyZMmNB3ZeUxdgW7SJzUsQfT5xi7u8/rz4rM7B+AhyquqKxQgOHD4eSTE1uliGSIOvZgKt0rZlK3by8HtlVWTjeaTkAkburYg6l0r5iVZjYTcOBV4IsVV1SmYJeUmdkU4D5gInAUWOPu30m3qoioYw+momB396uTKuQ92tu1D7ukrQv4r+6+2cxGAZvM7FF335F2YVFQxx5Mtnd3VMcuKXL3VnffXLq+H9gJTE63qoioYw9GwS7SD2bWCJwN/LqH+05sV14pUsceTDaD/cgR2LtXQzGSCWY2EngAuNHd3zr2/hPelVeK1LEHk81g37ev+GSrY5eUmVktxVBf5+7r064nKurYg8lmsOuoU8kAMzPgH4Gd7v7ttOuJjjr2YBTsIr07D7ga+HiQGUwHO3XswSQxu2PyysGuMXZJkbs/AWjawVDUsQeTzY5d88SIxE8dezDZDHYNxYjETx17MNkN9qFD4ZRT0q5EREJRxx5MNoO9vb3YreusKiLxUsceTDaDXUedisSv3LEr2BOnYBeRdJQ7dg3FJE7BLiLpUMceTDaDXVP2isRPHXsw2Qt2d3XsIoOBOvZgshfsBw7A4cMKdpHYqWMPJnvBrukERAYH7e4YTPaCXdMJiAwOOkApmOwFu6YTEBkc1LEHo2AXkXSoYw8mu8GuMXaRuKljDyZ7wd7eXpwjZuzYtCsRkZDUsQeTvWAvFGDMmHeedBGJkzr2YLIZ7BpfF4mfOvZgshfsmk5AZHBQxx5MxcFuZjeY2fNmtt3MVlZckTp2kcFBHXswFZ3M2sz+BFgInOXub5vZ+yquqFCAj3604tWISMapYw+momAH/hz4lru/DeDub1RckTr2xB0+fJiWlhY6OjrSLiVRdXV1NDQ0UFtbm3YpMhDq2IOpNNjPAC4ws78BOoAvu/vTPS1oZsuAZQBTp07teW0dHXDwoMbYE9bS0sKoUaNobGzEIjndoLtTKBRoaWlh2rRpaZcjA6GOPZg+g93MHgMm9nDX8tLPjwX+CJgD/JOZTXd3P3Zhd18DrAFoamp6z/2AjjoNpKOjI6pQBzAz6uvraWtrS7sUGSh17MH0GezuPq+3+8zsz4H1pSD/VzM7CowHBvZuU7AHE1Ool8X4mAYVdezBVLpXzP8CPg5gZmcAw4D2Aa9N0wmIDB460UYwlQb7vcB0M9sG/BBY2tMwTL9pyt5ojRw5Mu0SJGt0oo1gKvrw1N07gc8mVIuGYkQGE3XswVS6V0yyFOzh3XgjbNmS7DpnzoQ77+zXou7OV77yFR5++GHMjK997WssXryY1tZWFi9ezFtvvUVXVxerV6/m3HPP5brrrqO5uRkz49prr+VLX/pSsrVLetSxB5O9YB85Ek46Ke1KJJD169ezZcsWtm7dSnt7O3PmzGHu3Lncf//9fPKTn2T58uUcOXKEQ4cOsWXLFnbt2sW2bdsA2LdvX8rVS6LUsQeTrWBvb1e3Hlo/O+tQnnjiCZYsWUJNTQ2nnnoqH/vYx3j66aeZM2cO1157LYcPH+ayyy5j5syZTJ8+nZdffpkbbriB+fPnc9FFF1W9XjO7GPgOUAPc4+7fqnoRsSp37F1d6dYRoWxNAqajTqPX22frc+fOZePGjUyePJmrr76a++67j7Fjx7J161YuvPBC7rrrLj7/+c9XtVYzqwHuAj4FfARYYmYfqWoRMRs5sjhF9/PPp11JdLLVsSvYozd37lzuvvtuli5dyp49e9i4cSOrVq3itddeY/LkyXzhC1/g4MGDbN68mUsuuYRhw4Zx5ZVXctppp/G5z32u2uWeA7zo7i8DmNkPKc6NtKPahURpyBA47zx44AFYtAiGD3+nix/MJk6E6dMrWkX2gr3CByTZdvnll/Pkk08yY8YMzIyVK1cyceJE1q5dy6pVq6itrWXkyJHcd9997Nq1i2uuuYajpTHYb37zm9UudzLwu27ftwD/odpFRO2882DDBkhhmC2z/uIv4K67KlpFtoJ93jw466y0q5AADhw4ABSPFl21ahWrVq161/1Lly5l6dKl7/m5zZs3V6W+XvR0aOt7xpL6NQ+S9Oyaa6CtDebPL36IWsFhMNGYMqXiVWQr2FevTrsCke5agO7vsgZg97EL9WseJOnZxInw7W+nXUV0NKAl0rungdPNbJqZDQOuAn6Sck0ifcpWxy7BuHt0k2ZVMntFP9ffZWbXA/9CcXfHe919e9CNiiRAwT4I1NXVUSgUqK+vjybcy/Ox19XVhd7Oz4CfBd2ISMIU7INAQ0MDLS0t0c1dXj6Dkoi8m4J9EKitrdVZhkQGEX14KiISGQW7iEhkFOwiIpGx0LuM9bhRszbgtYCbGE8lp+jL57b1mN/xAXefUO1i4Liv7TSfn2Oplp7loZZ+vbZTCfbQzKzZ3ZsG07b1mLMtS7Wqlp7FVIuGYkREIqNgFxGJTKzBvmYQbluPOduyVKtq6Vk0tUQ5xi4iMpjF2rGLiAxaCnYRkcjkMtjN7F4ze8PMtnW7bZyZPWpmL5Qux3a776tm9qKZPW9mn0x4u6vM7Dkze9bMHjSzMaXbG83s/5nZltLXdwe63eNse4WZ7eq2jUu63RfyMf+o2zZfNbMtpduTfsxTzOz/mtlOM9tuZn9Zuj34c50kM7u4VM+LZnZzCtt/1cx+U3pOmku39fo7THjbqbxX+1lH8PdPL7WEf127e+6+gLnALGBbt9tWAjeXrt8M/G3p+keArcBJwDTgJaAmwe1eBAwtXf/bbttt7L5coMe8AvhyD8sGfczH3H8HcEugxzwJmFW6Pgr4t9JjC/5cJ/gYakp1TAeGler7SJVreBUYf8xtPf4OA2w7lfdqP+sI/v7ppZbgr+tcduzuvhHYc8zNC4G1petrgcu63f5Dd3/b3V8BXqR49vlEtuvuj7h7V+nbpyiePi1xvTzm3gR9zGVmZsBngB8MZN392Haru28uXd8P7KR4gungz3WCzgFedPeX3b0T+CHFOtPW2+8wUWm9V/tZR2+Cvo6q8brOZbD34lR3b4XiLw54X+n2ns40PzlQDdcCD3f7fpqZPWNmvzCzCwJt8/rSMNC93f51q9ZjvgB43d1f6HZbkMdsZo3A2cCvycZz3V9ZqMmBR8xskxVPvA29/w6rIUvPX5rvn2Cv65iCvTf9OtN8xRsxWw50AetKN7UCU939bOAm4H4zOyXhza4GTgNmlrZ3R7mcHpYNsV/rEt7drQd5zGY2EngAuNHd3zreoj3clvb+vFmo6Tx3nwV8CvjPZja3ytvvr2r/rlJ9/4R8XccU7K+b2SSA0uUbpdv7dab5SpjZUuDTwH/00qBY6d+mQun6JorjYmckuV13f93dj7j7UeAfeOffs2o85qHAFcCPutWT+GM2s1qKL/517r6+dHNqz/UApF6Tu+8uXb4BPEjxddLb77AaMvH8pfz+Cfq6jinYfwIsLV1fCvxzt9uvMrOTzGwacDrwr0lt1MwuBv4KWODuh7rdPsHMakrXp5e2+3JS2y2td1K3by8Hyp/4B33MJfOA59y9pVs9iT7m0hj+PwI73f3b3e5K5bkeoKeB081smpkNA66iWGdVmNkIMxtVvk7xw/5t9P47rIZMPH9pvX+q8rpO6pPean5R/Pe/FThM8a/ZdUA98HPghdLluG7LL6fYPT4PfCrh7b5IcfxrS+nru6VlrwS2U/w0ezNwaYDH/H3gN8CzpSd/UjUec+n27wF/dsyyST/m8yn+y/lst9/vJdV4rhN+vV5Ccc+Hl4DlVd729NLzsbX03Cwv3d7r7zDh7afyXu1nHcHfP2m9rjWlgIhIZGIaihERERTsIiLRUbCLiERGwS4iEhkFu4hIZBTsIiKRUbCLiETm/wPVEfhyGhe7pwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "guess_W = train(X, Y, 5, 200)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "def which_type(x, y, threshold=0.67):\n",
    "    X = np.array([x, y, 1]).reshape((3, -1))\n",
    "    f_x = f(X, guess_W)\n",
    "    h_x = g(f_x)\n",
    "    color = 'blue' if h_x < threshold else 'green' \n",
    "    plt.scatter(f_x, h_x, color=color)\n",
    "    \n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAADnCAYAAAC9roUQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de1hTZ74v8O9Kwh1EkYtgQMEAAoJU46W2tVfFK3b0mT1021Mt7VCL3e3e7Wm7e+Y4vUx9tDN7z2lHpzqMrWM7rUw7veCuiLdqxzqtCNZLxQsqWkBAUJA7ISvv+QOJIHdhZSXw/TwPT5J3vUl+WQlfFm/etZYkhAAREdmGRu0CiIiGEoYuEZENMXSJiGyIoUtEZEMMXSIiG9L1sJxTG8huzZkzB1lZWWqXQdQZqasF3NIlh1VRUaF2CUR9xtAlIrIhhi4RkQ0xdImIbIihS0RkQwxdIiIbYuiS4pKTk+Hv748JEyZ0ulwIgWeffRYGgwFxcXE4cuSIjSsksh2GLilu+fLl3c6n3bFjB/Lz85Gfn4+0tDQ8/fTTNqyOyLZ62jmCqN9mzpyJixcvdrk8IyMDjz32GCRJwvTp01FVVYWSkhIEBgbarkjqkmwRaDLLMJktaDJbYDJbYJItaJYtaDYLNFssMMsCZosFskXALIuWS4uARbRcv3kJWISApc11gZb/dlrbWm8LAQi0XsJ6Gzeut2rtC9zcm6tt31v7t9Wu+ZZO4/w9sSh+dD/WXOcYuqS64uJiBAcHW2/r9XoUFxd3GrppaWlIS0sDAJSXl9usRkclhEBVfTMqaptwtc6Eazd+rjc0o7qhGdWNzahpNKOm0Yy6JjPqTDIaTGbUm2Q0NMtobJbRLA+dHVOlNvuRzYoKYOjS4NTZgfQlqfO9KFNSUpCSkgIAMBqNitblCIQQuFpnwoXyOhRU1OKna/UormxAUWUDSqsbcaW6CSbZ0ul9XXQaDHNzgperDl4uOrg76zB6uDM8XLRwc9LCzVkLVyctXHVauDhp4KzVwFnX8uOi08BJ2/Kj00pw0rRc6jQSNJqW2xoNoNVI0EotbVpJglYjQZIAjSTd+Gl5ryUJ0N64lCBB0rTsRytJ0o3LG+1tPhatbTevt7bfaANu6d/lnrk2xdAl1en1ehQWFlpvFxUVISgoSMWK7JPFInChohZHfqpC3uVqnC6txunSGlTVN1v7aDUSRg1zhX6EG4xjRiDA2xUBXq7w9XLBSA9n+Nz48XZzgquTVsVXM3QxdEl1iYmJWL9+PZKSknDo0CF4e3tzPBctW7GnS2tw8FwFDp6rQO6lSlQ3mgEA7s5aRI7ywtwJgQj390SYnwfCfD0RNNwVOi2/H7dnDF1S3COPPIL9+/ejoqICer0er7/+OpqbW7bOVqxYgXnz5iEzMxMGgwHu7u7YvHmzyhWrxyxb8P2Fa9jxYwl25ZWhvKYJABDm54H5cYG4I2QEJoUMR5ivJzQa+/h3mfpG6uHElENnBJ0cjtFoRE5OjtplDIizZTVIzy7El0eLca3OBHdnLe6P9Md9kX64y+CLoOFuapdIfdPlX0Ru6RKpxGIR2JVXij8fKEDupUo4aSXMjh6FxPgg3BvhxzHXQYqhS2RjZtmCL49exob953C+vA5jRrrjV/OisHjSaIz0dFG7PFIYQ5fIhg7kl+M3X+XhbFktxo/ywrpH7sC82EBoOT47ZDB0iWyg8Fo9Xtt2EntPX0GIjzs2LJ2EORNG2c3cUbIdhi6RgoQQ+NvhQvzmqzxIkoT/nDsej981Fi46jtcOVQxdIoWU1zTh5c+O4+vTVzBj3Ej87ucTMZqzEIY8hi6RAk4UXUfKhzm4VmfCawuj8didYzmvlgAwdIkGXMbRYrz09+Pw9XTB56kzEBPkrXZJZEcYukQDRAiBdV+fw+93n8XUsT5499FJ8OUUMLoFQ5doAAgh8NudZ7Bh/3ksnjQaaxfHwVnHYyBQRwxdon4SQuDN7afw3rcF+NdpIXhz0QSO31KXGLpE/bT6RuAunzEWry6M5txb6hb//yHqh/e+LcAmBi71AUOX6DZl/ViCN7fnYU7MKKxawMCl3mHoEt2GIz9V4rn0o4gPHo63k+J57ATqNYYuUR9drW3Cig9zETDMFZseM/IQjNQn/CKNqA8sFoHnPzmGqoZmfPn4VB6KkfqMW7pEffCnf1zAN2fL8esF0YgOGqZ2OeSAGLpEvZR76Rr+a9cZzI8NxNJpIWqXQw6KoUvUC/UmM55LP4rRw92wZkksZyrQbeOYLlEv/H7XWRRVNuCTp+7EMFcntcshB8YtXaIeHCuswvsHC7B0WgimhvqoXQ45OIYuUTeaZQte/uw4/Lxc8PLc8WqXQ4MAhxeIupH2jws4XVqDtP81mcMKNCC4pUvUhZLrDVj3dT7mThiF2TGj1C6HBgmGLlEX/nvXWVgswP+ZF6V2KTSIMHSJOnHy8nV8dqQIj981FsE+7mqXQ4MIQ5foFkIIrN5+CsPdnJB6v0HtcmiQYegS3WLfmSv45/mreO7BcHi78cszGlgMXaI2ZIvAmszTCPX1wL9OG6N2OTQIMXSJ2sg8UYL8K7V4YXYETyxJiuCnihSXlZWFyMhIGAwGrF27tsPy69evY+HChZg4cSJiYmKwefNmFapsOWzj+q/PweDvibkTAlWpgQY/hi4pSpZlrFy5Ejt27EBeXh62bt2KvLy8dn3++Mc/Ijo6GseOHcP+/fvxwgsvwGQy2bzWXXmlOFNWg2fuN/BMEKQYhi4pKjs7GwaDAWFhYXB2dkZSUhIyMjLa9ZEkCTU1NRBCoLa2Fj4+PtDpbLuzpBAC674+h7Ej3bEgjlu5pByGLimquLgYwcHB1tt6vR7FxcXt+jzzzDM4deoUgoKCEBsbi3feeQcaTecfzbS0NBiNRhiNRpSXl/e7vtTtqdC9oYPHq9Nw8nI1vHy+g07LXwtSDj9dpCghRIe2W49Fu3PnTsTHx+Py5cs4evQonnnmGVRXV3f6eCkpKcjJyUFOTg78/Pz6VVvq9lRsyNkA2SLDuzkJZqkUXxX+GqnbU/v1uETdYeiSovR6PQoLC623i4qKEBQU1K7P5s2bsXjxYkiSBIPBgNDQUJw+fVrx2tJy0wAALpYYuIhIXNd9BkiytZ1ICQxdUtSUKVOQn5+PgoICmEwmpKenIzExsV2fkJAQ7N27FwBQVlaGM2fOICwsTPHaZCEDALzMiZBRgzrt1+3aiZTAQzuSonQ6HdavX4+EhATIsozk5GTExMRg48aNAIAVK1Zg1apVWL58OWJjYyGEwFtvvQVfX19F63rog4cAAFqLP9wt01Gt+xxCalL0OYkAQOpszK2NbhcSqcloNCInJ6fP93vog4ewt6Bly3p4czKGmReh2OUJyJoKax/xKj/61C9dzjnk8AINOa2BKwlXeJlno157sF3gaiWtWqXREMDQpSHLQ34QGniiWrutXXvK5BSVKqKhgKFLQ8pHJz5quSIkDDMvRJN0FibNzZkSHk4eeHf+uypVR0MBQ5eGlGVfLAMAuFri4CT0qNFtazf69qeFf1KpMhoqGLo0ZKRuT7VOB/OUE25MEztoXT7SbSSWxi5VqzwaIhi6NGRsyNkAANCIYXCX70Sddh8gNVuXvzP3HbVKoyGEoUtDjof5AUhwQq1uZ7t2buWSLTB0aUho3RkCAvCUZ6NJOo1mzSXrcmeNs0qV0VDD0KUhoXVuroslCs4iBDW3bOW+//D7apRFQxBDl4YUTzkBFtSjXnugXTuHFshWGLo06LUeqlES7nCX70ad9hsIqdG63MPJQ63SaAhi6NKg1zprwUO+Cxq4ola3u91yzs0lW2Lo0pDhId+PZqkIJulsu3YOLZAtMXRpUGvd7Vdr8YOrJQ512v3t9kB7MPRBdQqjIYuhS4Na626/HvK9ANCyQ0Qbex7bY/OaaGhj6NKg9dGJj1p2+xUtQwuNmpMwa8qsyzk3l9TA0KVB61d7fwUAcBJhcBZjWoYW2uDcXFIDQ5cGrUsnglr2QDM/AIFmzs0lu8DQpUEpNRXAsCIAGrjLM9GgyYFFqm1ZKAAteHYIUgdDlwalDRsAeBfB1RIHHXxQp9vfbvmWxVtUqYuIoUuDzkc3Tg6B6yFwl++BBfVo0By+2eF6CIcWSDUMXRp0fvWrG1e+Xg13+S7Uaw9BSKaWNpM7dN+/rFptRAxdGnQu3Thio2vNLGjhifrGM4CQgKoxwP9swF+eSVW3QBrSdGoXQKQUj/ElsDTp0LDuS0C++cXZUo4skIq4pUuDSmrrRqzGAreIUtSfDWgXuA9yr19SGUOXBpU/3ThgmNvYcmhdzag/Hdhu+R7u9UsqY+jSoGKxtFy6jy+B3KhDw0U/dQsiugVDlwYN69CCVoZ7RBkazo4CLDc/4k8/rU5dRG0xdGnQ2Lix5dIttAIaFzPqbhlaePddFYoiugVDlwYNIVou3SNKITfq0HjJV92CiDrB0CXFZWVlITIyEgaDAWvXru20z/79+xEfH4+YmBjce++9fX6Oh26cYR0aC9wMZWg4F9BuaCE6+nYqJxp4nKdLipJlGStXrsTu3buh1+sxZcoUJCYmIrpNClZVVSE1NRVZWVkICQnBlStX+vw8e1vOsA7X4GvQujWj/syodstPnuzXyyAaMNzSJUVlZ2fDYDAgLCwMzs7OSEpKQkZGRrs+H3/8MRYvXoyQkBAAgL+//20/n3tECSwmLRrbzFqQpG7uQGRjDF1SVHFxMYKDg6239Xo9iouL2/U5e/YsKisrcd9992Hy5Mn44IMPuny8tLQ0GI1GGI1GlJeXAwBGjGhdKuAWUYaGC34Q5ps7RKxYMWAvh6jfOLxAihKt3261Id2y6Wk2m5Gbm4u9e/eioaEBd955J6ZPn46IiIgO901JSUFKSgoAwGg0AgCqqlqWuYyuhM6zCZVn2w8tcNYC2ROGLilKr9ejsLDQeruoqAhBQUEd+vj6+sLDwwMeHh6YOXMmjh071mnodsc9ohTCrEHD+ZvDExr+L0d2hh9JUtSUKVOQn5+PgoICmEwmpKenIzExsV2fRYsW4cCBAzCbzaivr8ehQ4cQFRXVx2cScI8oRcNFXwiTk7X1qacG4EUQDSBu6ZKidDod1q9fj4SEBMiyjOTkZMTExGDjjT0ZVqxYgaioKMyZMwdxcXHQaDR48sknMWHChD49j5N/NXTDG1D1z/B27RxaIHsjdTbm1ka3C4nUZDQa8R//kYNHHwW87z4D7zvPoWj9LFgabp5avfuPN5Fiupwzw+EFcljXrgHJyS3X3cPL0FTk0y5wPTxUKoyoGwxdcljFxYDJBOi86+HsX4P6/IB2y1sP80hkTxi65LBMN0575hZeCgBoyG8/VYxniCB7xNAlh+ceXgbTFS+Yr7tb28aMUbEgom4wdMkhtR47V+Nmgov+WoehhdWrVSiKqBcYuuSQ0tJaLt3GlUHScGiBHAdDlxySLLdcuoeXwVztClPZMOsyT0+ViiLqBYYuOTTX0PIbQws3p0W2nkGCyB4xdMlhSS7N0DhZOLRADoWhSw5pzBhA42KGpVGHxkKfdu1E9oyhSw7pN28KaFybW44oduO0PO7unLVA9o+hSw5p/N2VkDQCHlUBkKSWLdy0NA4tkP3jUcbIIe3OK4ME4MRuP3i5ql0NUe9xS5ccjhACu/PK4OGig5erU893ILIjDF1yOOfLa1FQUYdhbgxccjwMXXI4u/LKAADDXDk6Ro6HoUsOZ3deGWJHe8NJy48vOR5+asmhXKlpxNHCKsyKDui5M5EdYuiSQ9l76gqEAEOXHBZDlxzK7rwy6Ee4YfwoL7VLIbotDF1yGHVNZnx7rgKzo0dBkro87x+RXWPoksP4x9lymMwWzI7h0AI5LoYuOYxdeWUY4e4E45gRapdCdNsYuuQQmmUL9p4qwwPjA6DjVDFyYPz0kkM4XHAN1Y1mDi2Qw2PokkPYlVcGVycNZob7qV0KUb8wdMnuCSGw62Qp7gn3g5uzVu1yiPqFoUt27+Tlaly+3sgdImhQYOiS3duVVwaNBDw43l/tUoj6jaFLdm/nj6UwjvXBSE8XtUsh6jeGLtm1C+W1OFNWg7kTRvXcmcgBMHRJcVlZWYiMjITBYMDatWu77Hf48GFotVr8/e9/t7btPNly7NyEGIYuDQ4MXVKULMtYuXIlduzYgby8PGzduhV5eXmd9nv55ZeRkJDQrj3rZCkm6r0RNNzNViUTKYqhS4rKzs6GwWBAWFgYnJ2dkZSUhIyMjA791q1bhyVLlsDf/+aXZZerGnCssAoJHFqgQYShS4oqLi5GcHCw9bZer0dxcXGHPl988QVWrFjRrn3XyVIAwJw2QwtpaWkwGo0wGo0oLy9XsHIiZTB0SVFCiA5ttx6W8d///d/x1ltvQattv+PDjh9LERHgiTA/T2tbSkoKcnJykJOTAz8/7p1Gjodn9iNF6fV6FBYWWm8XFRUhKCioXZ+cnBwkJSUBACoqKpCZmYkGocPhi0545n6DTeslUhpDlxQ1ZcoU5Ofno6CgAKNHj0Z6ejo+/vjjdn0KCgqs15cvX44FCxbAHDIJltwTHM+lQYehS4rS6XRYv349EhISIMsykpOTERMTg40bNwJAh3HcVjt+LEWwjxuiA4fZslwixUmdjbm10e1CIiVU1pkwZfUePHFPKF6ZG9VlP6PRiJycHBtWRtRrXZ5Pil+kkd3ZebIUZovAwrignjsTORiGLtmd7SdKMGakO2KCOLRAgw9Dl+zK1dom/PP8VSyIC+QZf2lQYuiSXck6WQrZIjA/lkMLNDgxdMmubD9egjA/D0QFeqldCpEiGLpkN8prmvD9hatYEMuhBRq8GLpkN7J+LIFFAPM5a4EGMYYu2Y3/OV4Cg78nIkdxaIEGL4Yu2YWiynpkF1zDw/HcyqXBjaFLdiHj6GUAwKL40SpXQqQshi6pTgiBz48UYepYHwT7uKtdDpGiGLqkuh+Lq3G+vA4P38GtXBr8GLqkui9+KIazVoP5sYFql0KkOIYuqcosW7Dt2GU8MN4f3u5OapdDpDiGLqnq23MVqKht4tACDRkMXVLVFz8Uw9vNCfeP5/nOaGhg6JJqrtc3I+vHUiyIC4SLTtvzHYgGAYYuqSbjWDGazBYkTQlRuxQim2HokiqEENiaXYiYoGGI1XurXQ6RzTB0SRXHi67jVEk1kqZyK5eGFoYuqSL9cCHcnLRYxGMt0BDD0CWbq2syY9vRYsyPC8QwV87NpaGFoUs2t/14CepMMh6ZGqx2KUQ2x9Alm/s4+ycY/D0xKWSE2qUQ2RxDl2zqWGEVjhZWYem0EJ6Sh4Ykhi7Z1OaDBfBy0eHnRg4t0NDE0CWbKatuxFfHS/BzYzA8XXRql0OkCoYu2cxfv78EWQgsnzFW7VKIVMPQJZtobJbx0aGf8FBUAEJG8uwQNHQxdMkmth29jGt1JiTfFap2KUSqYuiS4nbs2IGX388Cqoqx/5M/d1j+0UcfIS4uDnFxcZgxYwaOHTumQpVEtsHQJUXJsoyVqzdAeAdh7eOzkZ6+FXl5ee36hIaG4ptvvsHx48exatUqpKSkqFQtkfIYuqSoQ4cOwfmORQj2ccMSYwiSkpKQkZHRrs+MGTMwYkTLjhLTp09HUVGRGqUS2QRDlxS171QpGj1GIfU+A5y0Guj1ehQXF3fZ/7333sPcuXO7XJ6Wlgaj0Qij0Yjy8nIlSiZSFCdLkmKEENh9WQcXuR5LJumt7V3tibZv3z689957+Pbbb7t8zJSUFOvwg9FoHNiCiWyAW7qkmO8uXMXFOi28Lx+Cs67lo1ZUVISgoI6Hczx+/DiefPJJZGRkYOTIkbYulchmGLqkmD/szYe/lwvKD21DQUEBTCYT0tPTkZiY2K7fTz/9hMWLF+PDDz9ERESEStUS2QaHF0gR/zhbju8vXMOvF0Rj1B1vIyEhAbIsIzk5GTExMdi4cSMAYMWKFXjjjTdw9epVpKamAgB0Oh1ycnLULJ9IMZIQorvl3S4k6oxsEZj/hwOoM5mx5/l7FTvTr9FoZDiTveryEHocXqAB91luEU6X1uDlOeN5anWiWzB0aUDVNZnxX7vO4I6Q4ZgfG6h2OUR2h6FLA+rPBy7gSk0T/u/8KB6knKgTDF0aMJerGvCnby5gfmwgJo/xUbscIrvE0KUBIYTAqi9/BAD859zxKldDZL8YujQgtp8owd7TV/DC7AgE+/B4uURdYehSv1XVm/DatpOI03vzrBBEPeDOEdRvq7efQmV9Mz5Ingadln/HibrD3xDql72nyvBpbhGemhmG6KBhapdDZPcYunTbLlc14IVPjyE6cBiefTBc7XKIHAJDl26LWbbg2a0/oNlswR+XToKrE/c8I+oNjunSbfn97rPIuVSJd5LiEerroXY5RA6DW7rUZ7vzyvDu/vNImhKMRfGj1S6HyKEwdKlPjhVW4d+2HsFEvTdeXRijdjlEDoehS71WeK0eT2w5DD8vF2xaNgVuzhzHJeorjulSr1TWmbBsczaaZYG/PT4Vfl4uapdE5JAYutSjq7VNWLrpEIoqG/DXJ6ZhnJ+n2iUROSyGLnXrSnUjlm46hMLKery3zIipoTx6GFF/9GtM99NPP0VMTAw0Gk2H06asWbMGBoMBkZGR2LlzZ6f3v3btGmbNmoXw8HDMmjULlZWV/SmnS7/4xS8QHx+P+Ph4jB07FvHx8Z32Gzt2LGJjYxEfH2+T03u/9tprGD16tLW2zMzMTvtlZWUhMjISBoMBa9euVbyuF198EePHj8eEqffgntc+Q1FlPTYvn4p7wv3a9bPF+urptQsh8Oyzz8JgMCAuLg5HjhxRpI5bFRYW4v7770dUVBRiYmLwzjvvdOizf/9+eHt7W9/fN954wya19fS+qLHOzpw5Y10P8fHxGDZsGN5+++12fWy1vpKTk+Hv748JEyZY23qbRQPyuyiE6O6nW3l5eeL06dPi3nvvFYcPH7a2nzx5UsTFxYnGxkZx4cIFERYWJsxmc4f7v/jii2LNmjVCCCHWrFkjXnrppZ6est+ef/558frrr3e6bMyYMaK8vFzxGlq9+uqr4ne/+123fcxmswgLCxPnz58XTU1NIi4uTpw8eVLRunbu3Cm+P3dFTP7NLmF46Uvx+EtvdtpP6fXV02ufPHmy2L59u5gzZ46wWCziu+++E1OnTlWsnrYuX74scnNzhRBCVFdXi/Dw8A7vy759+8T8+fNtUk9bPb0vaq2zVmazWQQEBIiLFy+2a7fV+vrmm29Ebm6uiImJsbb1Jov6+LvYZa72a0s3KioKkZGRHdozMjKQlJQEFxcXhIaGwmAwIDs7u9N+y5YtAwAsW7YMX375ZX/K6ZEQAp988gkeeeQRRZ9nIGVnZ8NgMCAsLAzOzs5ISkpCRkaGos9ZOWI8Hn3/MDxddHhhogVNRXmKPl9XevPaMzIy8Nhjj0GSJEyfPh1VVVUoKSlRvLbAwEBMmjQJAODl5YWoqCgUFxcr/rwDQa111mrv3r0YN24cxowZY7PnbGvmzJnw8Wk/TNabLBqo30VFpowVFxcjODjYeluv13f6gSwrK0NgYMt5tAIDA3HlyhUlyrE6cOAAAgICEB7e+XECJEnC7NmzMXnyZKSlpSlaS6v169cjLi4OycnJnf5L09t1ORBqm8x4+e/H8fJnJzA9bCQyVt6N7Vvfx9y5czvtr/T66s1rt+X66crFixfxww8/YNq0aR2Wfffdd5g4cSLmzp2LkydP2qSent4XtddZenp6lxs+aqwvoHdZNFDrrccv0h566CGUlpZ2aF+9ejUWLVrU6X1EJ6d1V/p8Wb2pc+vWrd1u5R48eBBBQUG4cuUKZs2ahfHjx2PmzJmK1fX0009j1apVkCQJq1atwgsvvID333+/XT+l1uWtdck+oWiM/xfAfQRS7zPg+VkReGvtGuh0OixdurTTx1BifbXVm9euxmetrdraWixZsgRvv/02hg1rf5S1SZMm4dKlS/D09ERmZiYefvhh5OfnK15TT++LmuvMZDJh27ZtWLNmTYdlaq2v3hqo9dZj6O7Zs6fPD6rX61FYWGi9XVRUhKCgoA79AgICUFJSgsDAQJSUlMDf37/Pz9XbOs1mMz7//HPk5uZ22ae1Rn9/f/zsZz9DdnZ2v0Okt+vvl7/8JRYsWNChvbfr8nbrulZnwv/bfRZ/PXQJIT7u+P2/TMTkMT7YsmULvvrqK+zdu7fLD5YS66ut3rx2pdZPbzQ3N2PJkiVYunQpFi9e3GF52xCeN28eUlNTUVFRAV9fX0Xr6ul9UXOd7dixA5MmTUJAQECHZWqtL6B3WTRQ602R4YXExESkp6ejqakJBQUFyM/Px9SpUzvtt2XLFgDAli1butxyHgh79uzB+PHjodfrO11eV1eHmpoa6/Vdu3a1+3ZTCW3H0b744otOn2/KlCnIz89HQUEBTCYT0tPTkZiY2O/nNpkteO/bAtz3u334OPsnLLtzLDKfvQeTx/ggKysLb731FrZt2wZ3985PvWOL9dWb156YmIgPPvgAQgh8//338Pb2tv6bqCQhBJ544glERUXh+eef77RPaWmpdesoOzsbFosFI0eOVLSu3rwvaq0zoPv/NtVYX616k0UD9rvY3bdsPX0L+Pnnn4vRo0cLZ2dn4e/vL2bPnm1d9uabb4qwsDAREREhMjMzre1PPPGEdaZDRUWFeOCBB4TBYBAPPPCAuHr1ak9PeduWLVsmNmzY0K6tuLhYzJ07VwghxPnz50VcXJyIi4sT0dHR4s03O//GfiA9+uijYsKECSI2NlYsXLhQXL58uUNdQrR82xweHi7CwsL6XVeDySy2/LNAzFizV4x5+Svx6KbvxZnS6nZ9xo0bJ/R6vZg4caKYOHGieOqppzrUZav11dlr37Bhg9iwYYOYPHmysFgsIjU1VYSFhYkJEya0m0WjpAMHDggAIjY21rqetm/fbq1NCCHWrVsnoqOjRVxcnJg2bZo4ePCg4nV19b60rUutdVZXVyd8fHxEVVWVtU2N9ZWUlCRGjRoldDqdGD16tNi0aVOXWdSP38Uuc1USnYxTtM3kvsc42aPS6434JKcQH3x3ERW1JkwKGY5/e1g8EdkAAAQwSURBVDAc90X42XQMdCAZjcYO88OJ7ESXv1TcI20QazDJ+ObsFXyaU4R9Z67AIoCZEX5IvW8cpoX6OGzYEjkyhu4gU17ThIPnKrArrxT7TpejoVmGn5cLVtw7Dr+YEowxI3nAcSI1MXQdXMn1Bhy5VIUjP1Xi4LkKnC5t+RLF19MFSyaPxtwJgZgW6sOz9BLZCYaug6hpbMbFinpcqKjF6dIanC6pRl5JNcqqmwAALjoNJo8ZgRcTInFPuC9igryh1XD4gMjeKLr5Y6u9uvrKnuoSQuB6QzMKKuqw6g9/QeaJEmw+WIA1O07hmY+P4GfvHsSU1XsQ+9ouLFz/LZ5LP4pNBy6g5HojZozzxasLo5Gx8i6ceC0BH/9yOlbeb0CcfviABm5/15dSB60pLy/vV11KsafPV1usq2+UqkvR2Qv2+u1yV3UJIWARgNligcUCyEJAlgXMFgtki0CzRaDZbIHZYoHJLNAsW2CSLTCZW36azDKazBY0mGQ0NstoaLag3mRGXZOMepMZNU1m1DaaUdPYjKqGZlQ3NON6QzOa5Y6r2UkrIWi4G0bf+An180CYrwdCfT0R6usBZ53thgv68z7KsoyIiAjs3r0ber0eU6ZMwdatWxEdHW3tk5mZiXXr1iEzMxOHDh3Cc889h0OHDvX42B4eHqirq7utupTkaJ97tQ3Sumw7e+FCeS1SPsxF5fSVeOj333S6+xxwS6KLjldb7ycAtD6EgLh5ve19hLD2a+1jETfuIQDLjUC1CIHa+15B9K+zbrZZBGQh0P3fn9sjSYCHsw5uzlp4uerg5aKDl6sTAr3dMMzNCcPdnTDSwxk+Hs547ZX/jb/9JQ3+Xi4Y4e4MzSAYHmh7kBAA1oOEtA3drg7AYqsJ+0S21O2W7pw5c0RFRUWfH7RZtqDkeiOamprg4nJ7p3XpNG46aZS6+INy62yotjcbGxrh5u52s026uVy6cUNq0y5JLc8itT6uJEFz47p1mSRBI9281EgSNJLUoY7ulJeXw8/Pr+eONtafuiorK1FdXW09otTVq1dRV1eHkJAQa59z585h1KhR8PRsOSPF2bNnodfrO90brry8HK2fyaampi6Pjaymwfg+Kmkw1pWbm7tTCDGns2XcOYIU9emnn2Lnzp3YtGkTAODDDz9EdnY21q1bZ+0zf/58vPLKK7j77rsBAA8++CB++9vfYvLkyd0+tr3+W0qEboYXOI+IFGXvB60hsjWGLinKng9aQ6QGztMlRel0Oqxfvx4JCQmQZRnJycmIiYnBxo0bAQArVqzAvHnzkJmZCYPBAHd3d2zevFnlqomUwzFdclgc0yU7xjFdIiJ7wNAlIrIhhi4RkQ0xdImIbKinL9KI7JYkSVld7fVDZK8YukRENsThBSIiG2LoEhHZEEOXiMiGGLpERDbE0CUisqH/D/D8/rctzvfUAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "sigmod()\n",
    "_, length = green_X.shape\n",
    "for index in range(length):\n",
    "    item = blue_X[:, index]\n",
    "    temp = which_type(item[0], item[1])\n",
    "\n",
    "for index in range(length):\n",
    "    item = green_X[:, index]\n",
    "    temp = which_type(item[0], item[1])\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
